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Report 

OF THE NATIONAL EARLY LITERACY PANEL 



The National Early Literacy Panel 

The National Early Literacy Panel (NELP) was convened in 2002 to conduct a synthesis of the 
scientific research on the development of early literacy skills in children ages zero to five. The 
National Institute for Literacy (NILE) acted as the lead agency in this project, in consultation 
with cooperating agencies from the Partnership for Reading. The National Center for Eamily 
Literacy (NCEL), working closely with NILE, coordinated NELP s work in the completion of 
the synthesis. 

NELP was established for the express purpose of summarizing scientific evidence on early literacy 
development and on home and family influences on that development. The panels primary 
purpose was to synthesize research to contribute to decisions in educational policy and practice 
that affect early literacy development and to determine how teachers and families could support 
young children’s language and literacy development. In addition, this evidence would be a key 
factor in the creation of literacy-specific materials for parents and teachers and staff development 
for early childhood educators and family-literacy practitioners. 

NCEL project staff generated a list of expert researchers in areas of reading, early literacy, 
language, cognition, English as a second language, pediatrics, special education, research 
methodology, and early childhood education. NILE and its partner agencies approved the 
potential panelists before invitations were extended to the researchers seeking their participation 
as members of the panel. The members of NELP and their affiliations are as follows: 

Timothy Shanahan (panel chair). Professor of Urban Education at the University of 
Illinois at Chicago (UIC) and Director of the UIC Center for Literacy. 

Anne E. Cunningham, Associate Professor in the Graduate School of Education and 
Director of the Joint Doctoral Program in Special Education at the University of 
California, Berkeley. 

Kathy Escamilla, Professor in the School of Education at the University of Colorado 
and affiliated with the BUENO Center for Multicultural Education at the University 
of Colorado. 

Janet E. Eischel, Professor of Pediatrics and Psychology at the State University of New 
York at Stony Brook and Director of Pediatric Medical Evaluation, Director of the 
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Division of Developmental and Behavioral Pediatrics, Director of the Stony Brook 
Reading and Language Laboratory, and Associate Director of the Pediatric Residency 
Training Program. 

Susan Landry, Michael Matthew Knight Professor in the Department of Pediatrics at 
the University of Texas Health Science Center at Houston, Chief of the Division of 
Developmental Pediatrics, and Director of the Children’s Learning Institute (formerly, 
the Center for Improving the Readiness of Children for Learning and Education 
[CIRCLE]) in the Department of Pediatrics. 

Christopher]. Eonigan, Professor in the Department of Psychology at Elorida State 
University and Associate Director of the Elorida Center for Reading Research. 

Victoria]. Molfese, professor and Ashland/Nystrand Chair in Early Childhood 
Education, at the University of Eouisville; and Director of the University’s Early 
Childhood Research Center. 

Chris Schatschneider, Professor of Psychology at Elorida State University and Associate 
Director of the Elorida Center for Reading Research. 

Dorothy S. Strickland, Professor of Reading and Samuel DeWitt Proctor Professor of 
Education at Rutgers University. 

NEEP met on 12 occasions between April 2002 and Eebruary 2006. During these meetings, 
they deliberated on the proper way to search, synthesize, and summarize the research evidence on 
early literacy learning. Various subcommittees from the panel also met on several occasions and 
carried on these conversations through conference calls and email. 

Although panelists made the analytical decisions and supervised the research efforts, NCEE 
staff carried out much of the NEEP work. This work included organizing the panel, conducting 
electronic and hand searches, locating articles, and coding the information from the original 
studies so that analyses could be conducted. Most notable in this regard were the efforts of the 
following NCEE staff members: 

Eaura Westberg, Director of Special Projects and Research 

Kelly Coots, Research Specialist 

Eisa Smith ]ackson. Research Specialist 

Michelle Parkerson, Administrative Support Specialist 

Akeel Zaheer, Vice President for Program Services 

Tony Peyton, Senior Director, federal Initiatives, Policy & Research 

Sharon Darling, President and founder. 
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Executive Summary 

OF THE REPORT OF THE NATIONAL EARLY LITERACY PANEL 



Christopher J. Lonigan 
Florida State University 

Timothy Shanahan 
University of Illinois at Chicago 



The National Assessment of Educational Progress reveals that 37 percent of U.S. fourth graders 
fail to achieve basic levels of reading achievement. The incidence of reading failure is even higher 
within low-income families, ethnic minority groups, and English-language learners. Large-scale 
studies have shown that young children — those entering kindergarten and first grade — vary 
greatly in their attainment of the early precursor skills that provide the launching pad for later 
literacy learning (West, Denton, & Germino-Hausken, 2000; West, Denton, & Reaney, 2000). 
What can be done in U.S. homes, preschools, and kindergartens to better prepare children to 
succeed in learning to read and write? 

In 1997, the U.S. Congress asked that a review of research be conducted to determine what 
could be done to improve reading and writing achievement. The resulting Report of the National 
Reading Panel: Teaching Children to f?e< 2 iY(NICHD, 2000) has been influential in helping to 
guide reading-education policy and practice in the United States. However, that report did not 
examine the implications of instructional practices used with children from birth through age 
5. To address this gap in the knowledge base, the National Early Literacy Panel (NELP) was 
convened. The panel was asked to apply a similar methodological review process to that used by 
the National Reading Panel (NRP) to issues of instructional practices for young children so that 
parents and teachers could better support their emerging literacy skills. 

NELP was appointed in 2002 and carried out its work under the auspices of the National Center for 
family Literacy (NCEL). Laura Westberg, director of special projects and research at NCEL directed 
the effort. The National Institute for Literacy (NILE) funded the panel’s work in consultation with 
the National Institute for Child Health and Human Development (NICHD), the U.S. Department 
of Education, and the Office of Head Start in the U.S. Department of Health and Human Services. 
The panel included the following experts in literacy and early childhood education: 

Anne Cunningham, University of California, Berkeley 

Kathy Escamilla, University of Colorado at Boulder 

Janet Eischel, State University of New York at Stony Brook 
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Susan H. Landry, University of Texas-Houston 

Christopher]. Lonigan, Florida State University 

Victoria Molfese, University of Louisville 

Chris Schatschneider, Florida State University 

Timothy Shanahan, NELP chair. University of Illinois at Chicago 

Dorothy Strickland, Rutgers University. 



Questions Addressed by the National Early Literacy Panel 

NELP s primary goal was to identify interventions, parenting activities, and instructional 
practices that promote the development of childrens early literacy skills. Toward that end, the 
panel posed the following four questions: 

1 . What are the skills and abilities of young children (age birth through five years or 
kindergarten) that predict later reading, writing, or spelling outcomes? 

2. Which programs, interventions, and other instructional approaches or procedures have 
contributed to or inhibited gains in children’s skills and abilities that are linked to later 
outcomes in reading, writing, or spelling? 

3. What environments and settings have contributed to or inhibited gains in children’s skills 
and abilities that are linked to later outcomes in reading, writing, or spelling? 

4. What child characteristics have contributed to or inhibited gains in children’s skills and 
abilities that are linked to later outcomes in reading, writing, or spelling? 

NELP adopted a methodology that allowed for the identification and selection of published 
studies relevant to the panel’s questions, a coding system that allowed for the combination and 
comparison of studies, and an appropriate method of statistical analysis. Electronic searches 
were conducted using PsycINEO and the Education Resources Information Center (ERIC), 
and these were supplemented with hand searches of major research journals, reference checks of 
past literature reviews, and nominations from leading experts in the field of early literacy. These 
search procedures yielded more than 8,000 potential articles that were screened to determine their 
relevance to the research questions and their consistency with all selection criteria established by 
the panel. This led to the identification of approximately 500 research articles that were used in 
the meta-analyses conducted by the panel. The meta-analyses summarized both correlational data 
showing the relationships between children’s early abilities and skills and later literacy development 
and experimental data that showed the impact of instructional interventions on children’s learning. 



Key Findings of the National Early Literacy Panel 

Identification of the Domain of Early Literacy Skills 

The panel set out first to establish which early skills or abilities could properly be said to 
be the precursors of later literacy achievement. This was important because, without such 
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a determination, it would be impossible to ascertain what programs or practices were most 
effective, because, even in the best of circumstances, most young children develop few 
conventional literacy skills before starting school. To identify the essential early skills or abilities 
relevant to later literacy development, the panel searched for published scientific studies that 
could provide correlational evidence showing the relationship between early skill attainment and 
later literacy growth in decoding, reading comprehension, or spelling. 

Conventional literacy skills refers to such skills as decoding, oral reading fluency, reading 
comprehension, writing, and spelling. The use of these skills is evident within all literacy 
practices, and they are readily recognizable as being necessary or useful components of literacy. 
The term conventional literacy skills is not widely used in the field but is adopted here to 
distinguish between these aspects of literacy that are clearly the focus of the reading, writing, 
and spelling instruction provided to elementary and secondary students and those earlier- 
developing precursor skills that may not themselves be used within literacy practice but that 
may presage the development of conventional literacy skills. Conventional skills can be thought 
of as being more sophisticated, mature, or later-developing manifestations of reading and 
writing, and they are to be contrasted with precursor, predictive, foundational, or emergent skills 
(all terms used in this report). The report sometimes uses, more generally, early literacy skills, 
which can refer to both precursor skills and the conventional literacy skills of preschool and 
kindergarten children. 

Conventional reading and writing skills that are developed in the years from birth to age 5 have 
a clear and consistently strong relationship with later conventional literacy skills. Additionally, 
six variables representing early literacy skills or precursor literacy skills had medium to large 
predictive relationships with later measures of literacy development. These six variables not only 
correlated with later literacy as shown by data drawn from multiple studies with large numbers of 
children but also maintained their predictive power even when the role of other variables, such as 
IQ or socioeconomic status (SES), were accounted for. These six variables include 

• alphabet knowledge (AK): knowledge of the names and sounds associated with 
printed letters 

• phonological awareness (PA): the ability to detect, manipulate, or analyze the auditory 
aspects of spoken language (including the ability to distinguish or segment words, 
syllables, or phonemes), independent of meaning 

• rapid automatic naming (RAN) of letters or digits: the ability to rapidly name a sequence 
of random letters or digits 

• RAN of objects or colors: the ability to rapidly name a sequence of repeating random 
sets of pictures of objects (e.g., “car,” “tree,” “house,” “man”) or colors 

• writing or writing name: the ability to write letters in isolation on request or to write 
one’s own name 

• phonological memory: the ability to remember spoken information for a short period 
of time. 
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An additional five early literacy skills were also moderately correlated with at least one measure 
of later literacy achievement but either did not maintain this predictive power when other 
important contextual variables were accounted for or have not yet been evaluated by researchers 
in this way. These additionally potentially important variables include 



• concepts about print: knowledge of print conventions (e.g., left-right, front-back) and 
concepts (book cover, author, text) 

• print knowledge: a combination of elements of AK, concepts about print, and 
early decoding 

• reading readiness: usually a combination of AK, concepts of print, vocabulary, memory, 
and PA 

• oral language: the ability to produce or comprehend spoken language, including 
vocabulary and grammar 

• visual processing: the ability to match or discriminate visually presented symbols. 



These 1 1 variables consistently predicted later literacy achievement for both preschoolers 
and kindergartners. Not surprisingly, these measures were usually more predictive of literacy 
achievement at the end of kindergarten or beginning of first grade than of later literacy growth. 
The report provides an analysis of the particular relations among these variables. For instance, 
oral language was found to play a bigger role in later literacy achievement when it was measured 
using more complex measures that included grammar, the ability to define words, and listening 
comprehension than when measured using only simple vocabulary knowledge. Also, childrens 
early PA — that is, their ability to distinguish among sounds within auditory language — ^was found 
to be an important predictor of later literacy achievement, expanding on earlier NRP findings. 



Instructional Practices That Enhance Early Literacy Skills 

The panel also set out to identify studies that employed experimental or quasiexperimental 
methods to determine the effectiveness of instructional strategies, programs, or practices in 
imparting conventional literacy skills or any of these precursor skills to young children. The 
panel did not set out to find evaluations of previously identified programs or interventions but 
searched for all such studies that had been published in refereed journals in the English language. 
The panelists then grouped the identified studies into five analytical categories. The categories of 
intervention and the number of studies within each category included the following: 

• Code-focused interventions (« = 78): Interventions designed to teach children skills 
related to cracking the alphabetic code. Most code-focused interventions included 
PA instruction. 

• Shared-reading interventions {n = 19): Interventions involving reading books to children. 
These interventions included studies of simple shared reading and those that encouraged 
various forms of reader-child interactions around the material being read. 
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• Parent and home programs {n = 32): Interventions using parents as agents of 
intervention. These interventions may have involved teaching parents instructional 
techniques to use with their children at home to stimulate childrens linguistic or 
cognitive development. 

• Preschool and kindergarten programs {n = 33): Studies evaluating any aspect of a 
preschool or kindergarten program. Ten studies in this category concerned one particular 
intervention (the Abecedarian Project). Other studies evaluated effects of educational 
programs, curricula, or policies, such as extended-year experience, on kindergartners. 

• Language-enhancement interventions (« = 28): Studies examining the effectiveness of an 
instructional effort aimed at improving young children’s language development. 

The code-focused instructional efforts reported statistically significant and moderate to 
large effects across a broad spectrum of early literacy outcomes. Code-focused interventions 
consistently demonstrated positive effects directly on children’s conventional literacy skills. Book- 
sharing interventions produced statistically significant and moderate-sized effects on children’s 
print knowledge and oral language skills, and the home and parent programs yielded statistically 
significant and moderate to large effects on children’s oral language skills and general cognitive 
abilities. Studies of preschool and kindergarten programs produced significant and moderate to 
large effects on spelling and reading readiness. Finally, language-enhancement interventions were 
successful at increasing children’s oral language skills to a large and statistically significant degree. 
Together, these findings suggest that there are many things that parents and preschools can do to 
improve the literacy development of their young children and that different approaches influence 
the development of a different pattern of essential skills. 

There is great interest in the idea of providing age-appropriate interventions. Ffowever, there were 
few important differences among these categories of study with regard to age; one important 
exception was in the area of language interventions, which showed greater effectiveness early 
on. Otherwise, when age-level comparisons were possible, the large and significant effects of 
the various interventions were obtained with groups of both younger and older children. This 
means that most of the types of instruction that are effective in kindergarten are very similar 
to those that can be used in preschool. Unfortunately, there have not been direct tests of age 
differentiation in early literacy instruction across kindergarten and preschool, and there are 
still too few studies of preschool literacy instruction to provide comparison results that can be 
embraced with a high degree of certainty. Future research into this issue could shed greater light 
on what, to some observers, may seem a surprising finding. 

Few interventions improved conventional literacy skills or the precursor skills most related 
to later literacy growth, the exception being code-focused interventions. One reason so few 
interventions were found to foster improvement in these measures is that few intervention studies 
with young children included measures of such outcomes. Generally, code-focused intervention 
studies included such measures, while studies of other instructional approaches did not. It is 
possible that some of these other approaches may also be effective in improving early literacy 
skills, but that can only be determined through studies employing such measures. Code-focused 
programs, book sharing, programs for parents to use at home, and language-enhancement 
instruction all improved children’s oral language skills. 
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The panel wanted to determine whether any child characteristics influenced the effectiveness of 
the instructional interventions. In most cases, the panel could not determine the role of childrens 
characteristics because of reporting limitations in the original studies. In general, however, 
variables, such as age, SES, and race, did not seem to alter the effectiveness of the various 
interventions, and it will take future research to determine whether certain interventions would 
be effective with particular groups of children. 

It should be noted that the interventions that produced large and positive effects on children’s 
code-related skills and conventional literacy skills were usually conducted as one-on-one or 
small-group instructional activities. These activities tended to be teacher-directed and focused on 
helping children learn skills by engaging in the use of those skills. Almost all of the code-focused 
interventions included some form of PA intervention. These PA activities generally required 
children to detect or manipulate (e.g., delete or blend) small units of sounds in words. Few of the 
interventions used rhyming activities as the primary teaching approach. Teaching children about 
the alphabet (e.g., letter names or letter sounds) or simple phonics tasks (e.g., blending letter 
sounds to make words) seemed to enhance the effects of PA training. 



Limitations 

The major limitation confronting any meta-analysis is the quality of the original studies that 
are being combined. All studies have varying degrees of weakness in their implementation and 
reporting. A basic premise of meta- analysis is that all studies on a particular issue would be 
unlikely to suffer the same problems and that the influence that such factors may have on results 
can therefore be analyzed and understood. The reality is that the various study-design features, 
demographic characteristics of participating children, and crucial elements of the educational 
environments are hopelessly confounded across studies. Therefore, meta- analysis provides clues to 
what might be influencing the effectiveness of an intervention but cannot provide the final word 
on such findings. 

It is impossible to be certain that any meta-analysis will identify all studies on a particular topic, 
and any study that is not included could provide information that would be at odds with the 
conclusions drawn. In this case, because the meta-analysis examined only the results of published 
studies, it is possible that a somewhat different picture could be derived if a broader net were cast. 

In this case, many substantive issues of great concern to educators and parents could not even 
be explored adequately because of limitations in the reporting of original studies. There are 
many theories, both naiVe and scientific, suggesting the likelihood of individual differences in 
instructional effectiveness that demographic characteristics might mitigate. This meta-analysis 
evaluated whether such variables as race or SES mitigated or moderated the effectiveness of the 
various interventions. Unfortunately, it was all too rare that the original studies had provided 
sufficient data to allow for unambiguous conclusions to be drawn. 

Future Research Directions 

The NELP report provides a rich set of findings about the relationship between early developing 
child skills and later literacy attainment and the effectiveness of interventions for helping young 
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children to progress toward successful literacy learning. The analyses carried out by the panel also 
reveal important gaps in the empirical research record that future research should address. 

The panel identified which early measures of childrens skills were predictive of later decoding, 
reading comprehension, and spelling achievement. Some of these variables — certain aspects 
of phonological processing, for example — have been shown in previous research to be causally 
connected to literacy achievement (i.e., if those skills are taught, children attain higher levels of 
literacy), but this is not true for all of these variables. Future research must determine whether 
enhanced early instruction aimed at improving skills, such as AK, concepts of print, or oral 
language development, would consistently lead to higher later attainments in literacy. 

The panel identified a wide variety of interventions that improved childrens early literacy skills, 
and one pattern that emerged was that the various categories of interventions had qualitatively 
different outcomes. For example, the code-oriented interventions improved childrens knowledge 
of phonology and print conventions, whereas shared-book interventions enhanced childrens 
language development. It is possible that some of these interventions would actually have 
a wider impact than what was determined here, but that will require that future studies of 
such interventions employ a wider range of outcome measures. In fact, this would be a useful 
research convention for early literacy-intervention research; if such studies would use a wider 
range of outcome measures, it would be possible to determine the breadth of impact that 
these interventions may have. Also, given the complementary findings for the various types of 
intervention, it would be helpful if researchers would undertake longitudinal studies of more 
complex interventions (such as combinations of the types of efforts that have worked in the past), 
making it possible to evaluate the long-term value of more ambitious and complete efforts to 
develop early literacy skills. 

Finally, the NELP report found few demographic differences in childrens learning patterns, 
and even those that were found were confounded. Future studies of early literacy skills should 
consider the possibly varied impact of early interventions, particularly on large and growing 
groups of children who struggle with literacy (such as second-language learners and children 
being raised in poverty). However, even if research studies are not designed to specifically answer 
such questions, it would be helpful if they would report their data separately for children from 
different demographic categories, as this would make it possible for future meta- analyses to make 
sense of any patterns that may exist. 



Conclusions 

The NELP report represents a systematic and extensive synthesis of the published research 
literature concerning children’s early literacy skills. It provides educators and policymakers with 
important information about the early skills that are implicated in later literacy learning, as 
well as information about the type of instruction that can enhance these skills. The results also 
identify areas in which additional research is needed. 

The meta-analyses conducted by the panel showed that a wide range of interventions had a 
positive impact on children’s early literacy learning. However, these positive results were due 
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to the nature and intensity of the instructional activities examined in the studies. There is 
now a clear need for translational research. Researchers or their agents delivered many of the 
interventions; examinations of more typical implementations of such programs within early 
childhood education are needed. Many of the high-impact instructional strategies involved 
activities and procedures different from those typically seen in early childhood classrooms. 

These interventions were usually delivered as one-on-one or small-group activities, they 
occurred frequently, and they were adult-directed. Few interpretable studies evaluated the 
effects of merely providing a literacy-rich or language-rich classroom environment. 

Finally, there were significant problems with the quality of much of the research in this area. 
Many studies used simple pretest-posttest designs, which provide no causally interpretable 
evidence, and studies often did not provide evidence that these groups were equivalent prior 
to an intervention or represented the same population. Often, there was evidence for group 
differences that existed before the start of the intervention. The panel was unable to rely on the 
data drawn from such badly designed studies, and they were excluded from all of the analyses 
reported here. These flaws do not allow appropriate postintervention differences to be attributed 
unambiguously to the intervention; neither do studies in which the intervention is confounded 
with other important factors that could be the source of any observed effects. Ultimately, 
building a larger and more comprehensive knowledge base concerning early literacy skill 
development and promotion will require more high-quality research. 
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Introduction 

The ability to read and write is fundamental to full participation in American society. Our nation 
of farmers and mechanics has been transformed into one in which economic, civic, and social 
success depend on educational attainment for all, particularly in literacy. The rapid influx of 
technology into our daily lives and the internationalization of the economic marketplace have 
raised the demand for a literate citizenry to the highest levels ever (Carnevale, 1991). 

Many Americans cannot read well enough to take full advantage of the benefits of society — or to 
contribute fully to its sustenance (Kirsch, Jungeblut, Jenkins, & Kolstad, 1993). Those who are 
low in literacy are paid less, are more often out of work, are less likely to vote, are less informed 
about civic affairs, are less able to meet the health-care needs of their families, and are more likely 
to have trouble with the law or to become ensnared in other socially harmful activities. ' Literacy 
is implicated in virtually every sphere of our daily lives, no matter how mundane or profound — 
from following a prescription to taking part in a religious service, from sending an email to 
buying something over the Internet, from reading a sign for directions to reading a book to 
one’s children. 

In 1997, the U.S. Congress requested the appointment of a panel of scientists to review research 
on reading instruction to determine what could be done to improve reading achievement. The 
National Reading Panel (NRP) conducted a review of research on elementary and secondary 
reading instruction (NICHD, 2000), and its report has become the basis of new federal 
education laws designed to foster improved reading instruction from kindergarten to third grade. 

As critics have pointed out, NRP failed to examine what could be done during the preschool 
years to better prepare children for success in reading.^ This new report seeks to redress that 
important omission. This report, written by the National Early Literacy Panel (NELP), 

^ The statistics on employment, economic well being, voting, and health are drawn from Venezky, Kaestle, and Sum (1987). That 
report was based on data drawn from Kirsch and Jungeblut (1986). These findings were further confirmed by later analyses of 
the National Adult Literacy Survey, including Kirsch, Jungeblut, Jenkins, and Kolstad (1993) and Sum (1999). Statistics on the 
relationship of literacy to crime and incarceration are from Haigler, Harlow, O’Connor, and Campbell (1994), and the statistics 
on the relationship of literacy attainment to other social problems, including teen pregnancy, drug use, and violence are from 
Matson and Haglund (2000), Bennett, Brown, Boyle, Racine, and Offord (2003), and Kellam and Anthony (1998). 

^ As summarized in Shanahan (2004). 
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systematically examines the research on early literacy instruction to determine what best can be 
done to prepare young children for literate lives. 



Why Focus on Young Children 

More than one-third of Americas fourth graders read at levels so low they cannot complete their 
schoolwork successfully (Lee, Grigg, & Donahue, 2007). Reading achievement is particularly low 
for Latino and African American fourth graders, 56 percent and 60 percent, respectively, of whom 
read at those below-basic levels that do not even provide sufficient support to allow the completion 
of schoolwork (Lee, Grigg, & Donahue, 2007). National literacy assessments reveal that levels of 
literacy attainment are not much better for eighth or 12th graders or even adults,^ which is not 
surprising, given that beginning literacy is highly predictive of later literacy attainment. 

Several research studies have demonstrated that early cognitive and linguistic development 
predict later achievement — even much later. Various measures administered at the preschool 
and kindergarten levels reveal that patterns of preschool learning are closely linked with reading 
achievement in the primary grades (Scarborough, 1998). Young children who demonstrate oral 
language proficiency and early abilities in processing print do better in learning to read in first, 
second, and third grades (Scarborough, 2001). This means that learning achieved during these 
early years is likely to be sustained throughout the primary-school years and is an important basis 
for successful early performance in school. 

Success in literacy learning during the primary grades is even more indicative of later literacy 
achievement. Seventy-four percent of children who perform poorly in reading in third grade 
continue to do so into high school, further underlining the importance of preparing children to 
enter school ready to learn (Fletcher & Lyon, 1998). 

Before children enter elementary school, they must develop many linguistic and cognitive skills 
that will make later academic learning possible. By the age of five, however, children differ 
markedly in their success in reaching these developmental goals (Entwisle & Alexander, 1993), 
and these early differences reverberate throughout a child’s schooling, limiting or amplifying 
learning success. 

Differences in early achievement have been correlated with mothers’ levels of education and 
family economic success (Nord, Lennon, Liu & Ghandler, 2000). Not all children enter school 
equally prepared to learn to read, and those from the nation’s most economically disadvantaged 
families are the least likely to be well prepared to succeed — opening initial achievement gaps 
between rich and poor and black and white that are never spanned during a child’s schooling. A 
growing body of evidence shows that high-quality early education can have long-lasting benefits 
(Bowman, Donovan, & Burns, 2001; Shonkoff & Phillips, 2000), but these studies have not 
usually focused on early literacy learning outcomes. 

The 2000 U.S. census indicates that there are approximately 2 1 million children ages birth to 5 
residing in the United States (U.S. Gensus Bureau, 2002). Of these, approximately 5.5 million 

^ Statistics on eighth-grade reading levels are drawn from NAEP and NCES (2007); 12th-grade reading statistics are from 
Campbell, Hombo, and Mazzeo (2000); and adult-literacy statistics were drawn from Kirsch, Jungeblut, Jenkins, and 
Kolstad (1993). 
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are enrolled in some form of center-based preschool, including approximately 55 percent of four- 
year-olds (NCES, 2000). Almost a million of these children are enrolled in federally supported 
Head Start programs at an annual cost of $6 billion (HHS, 2008). This means that there are 
large numbers of young children — including young children from low-income families, already 
enrolled in preschool programs and child-care arrangements — who could benefit from enhanced 
preschool literacy preparation. 

Most children spend considerable time with their parents rather than in preschools or child- 
care arrangements (U.S. Census Bureau, 2002). Because parents, too, can provide children with 
excellent literacy preparation during these early years, information on what would benefit young 
children’s literacy development would be useful in those situations as well. 
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Overview 

The National Early Literacy Panel (NELP) was charged with conducting a research synthesis 
on early literacy development. This charge was not simply to complete a literature review but to 
engage in a systematic empirical study in which data are collected, analyzed, and evaluated in an 
objective and systematic way to determine answers to specified research questions. In that sense, 
a research synthesis is an independent research study in its own right that uses existing studies 
as the data for its analysis. As independent research studies, research syntheses include selection 
criteria for identification of relevant research, standards for judging the quality of research, 
operational definitions, and replicability of methods. 

Meta-analysis is increasingly being used to summarize research evidence for a variety of purposes, 
and meta-analytic procedures continue to develop. There are still controversies about how best to 
conduct and interpret sound meta-analytic work. NELP used methodology consistent with that 
used by the National Reading Panel (NRP) in the completion of its synthesis on reading and its 
implications for reading instruction. NELP consulted other key resources on research synthesis 
in order to improve on the original model and to better meet the specific inquiry demands of 
the research questions (Cooper & Hedges, 1994; Cooper, 1998; Shanahan, 2000). Generally, 
the procedures used by NELP sought to identify the most comprehensive set of obtainable data 
in an unbiased way and to analyze those data in a straightforward manner with a minimum of 
manipulation or recalculation of the original data. Although various innovative methods have 
been proposed for “correcting” original data (Hunter & Schmidt, 2004), it is not clear whether 
such procedures actually improve the validity of results, so they were not employed here. 

Simply put, meta-analysis is a procedure aimed at determining the average results of a collection 
of independent studies and of examining variations in those results to determine the reason for 
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those variations. This means that researchers must identify a population of studies that address a 
particular question, develop rules for systematically selecting which studies can be combined or 
compared, code key comparative information from the original studies, and analyze these results 
statistically to determine the size of an effect and which variations in study procedures, subject 
samples, or instructional circumstances are correlated with differences in these effects. 

This chapter provides a description of the research questions that NELP attempted to answer. It 
also describes the methods used to identify and select research studies, to code those studies, and 
to analyze the results. 

Research Questions 

To achieve the purposes of the project in synthesizing the research literature on early literacy 
development, NELP formulated four basic research questions: 

1 . What are the skills and abilities of young children (ages birth through five years or 
kindergarten) that predict later reading, writing, or spelling outcomes? 

2. Which programs, interventions, and other instructional approaches or procedures have 
contributed to or inhibited gains in children’s skills and abilities that are linked to later 
outcomes in reading, writing, or spelling? 

3. What environments and settings have contributed to or inhibited gains in children’s skills 
and abilities that are linked to later outcomes in reading, writing, or spelling? 

4. What child characteristics have contributed to or inhibited gains in children’s skills and 
abilities that are linked to later outcomes in reading, writing, or spelling? 

The panel decided that, to answer research questions 2, 3, and 4, research question 1 had to 
be adequately answered first. That is, to examine adequately the impact of environments, child 
characteristics, and interventions on early literacy skills and abilities, it was necessary to first 
determine what constitutes the early skills or precursors of reading, writing, or spelling. To 
do this, the panel had to determine what early abilities and skills were predictive of later 
outcomes in reading, writing, or spelling that would be appropriate to consider in answering 
these other questions. 

The first question will be answered in detail in Chapter Two of this report. Chapters Three 
through Seven will then address questions 2, 3, and 4 for a particular major type of intervention. 
Chapter Three will consider code-focused interventions (those that explicitly teach phonological 
awareness [PA], phonics, alphabet, or print awareness). Chapter Eour covers shared-reading 
interventions. Chapter Live focuses on parent and home programs. Chapter Six analyzes the 
effectiveness of preschool and kindergarten programs, and Chapter Seven considers language- 
enhancement interventions. Eor instance. Chapter Three considers whether code-focused efforts 
are effective in improving children’s literacy and literacy-related skills (question 2), then considers 
whether such programs are more or less effective under various instructional circumstances 
(question 3) and whether they work better with some types of children (question 4). All of the 
intervention chapters have that structure. 
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Methodology for Predictors 



Search Terms and Search Procedures 

Prior to conducting a systematic electronic search, project staff generated a list of search terms 
in nine categories (language, cognition, motivation, schooling, home and family, word learning, 
fluency, reading comprehension, miscellaneous^ that NELP reviewed, revised, and approved. 

To identify key terms, staff consulted such references as The Literacy Dictionary (Harris & 
Hodges, 1995), Preventing Reading Difficulties in Young Children (Snow, Burns, & Griffin, 1998), 
Handbook of Early Literacy Research (Neuman & Dickinson, 2001), and Handbook of Reading 
Research (Pearson, Barr, & Kamil, 2000, pp. 209-226). After NELPs review and revision of the 
terms and categories, a final list was completed with 284 key search terms identified across the 
nine categories. In addition to the nine categories, two categories were created for age group and 
literacy outcomes with a total of 67 key terms identified in these categories. (See Appendix 1 .A 
for a complete list of the superordinate categories and subordinate search terms.) 

Searches were conducted using both the Education Resources Information Center (ERIC) and 
PsycINEO databases. Each subordinate term within a category was linked using or statements 
to identify as many citations as possible of unique research articles that referred to that category. 
Then the category citations were linked with age identifiers and literacy outcomes to ensure the 
relevance of this information to the research questions. The goal was to generate as many records 
in each database for each of the nine topical categories. This search process resulted in 7,313 
records. Table 1 . 1 identifies the number of records generated for each superordinate term per 
search engine. 



Table 1.1. Records Generated per Search Engine 



Superordinate Term 


ERIC 


PsycINFO 

Total 


PsycINFO, 

1984-Present 


PsycINFO, 

1967-1983 


PsycINFO, 

1887-1966 


Language 


963 


427 


340 


75 


12 


Cognition 


518 


360 


259 


93 


18 


Motivation 


258 


97 


75 


16 


6 


Schooling 


1,363 


421 


306 


98 


17 


Home and family 


528 


213 


176 


32 


5 


Word learning 


384 


198 


163 


32 


3 


Fluency 


242 


64 


52 


11 


1 


Reading comprehension 


191 


103 


80 


21 


2 


Miscellaneous 


796 


187 


135 


44 


8 



Inclusion and Exclusion Criteria 

To ensure the identification of studies of high quality and relevance to the research questions, 
NELP formulated initial screening criteria and rationales that included the following: 

1 . The study must be published in English, because the panel did not have the resources to 
review articles in other languages. 
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2. The study must be published in a refereed journal. This ensures a minimum level of quality, 
in that such research had previously been evaluated by a panel of experts and was judged 

to be of sufficient methodological rigor to warrant publication. The reliance on published 
studies alone increases the chances that results will be overly affected by publication bias. 
What this means is that studies that have positive results are more likely to be published, 
and therefore, the average effects that result from an analysis of published studies will tend 
to be higher. Although this is true, there are not well-validated procedures for searching 
for unpublished studies in unbiased ways, and the expense — in terms of both time and 
money — were beyond the means of this project. Given both the potential for limitations and 
the difficulties and uncertainties in overcoming them, the research team decided to follow 
the precedent of the NRP procedures and the definitions of scientific-based reading research 
expressed in federal education law, by relying solely on published studies that had received 
some level of independent quality screening. It is assumed that the effect sizes (ESs) obtained 
here would probably be somewhat attenuated if unpublished studies could be obtained. 

3. The study must include empirical research that provided quantitative data on groups 
of children who represented the normal range of abilities and disabilities that would be 
common to regular classrooms (as opposed to special educational placements). The panel 
could examine only that research that analyzed data on children’s early literacy learning. 

To be included, these studies had to provide quantitative data describing children within 
a normal range of abilities and disabilities so that the statistical data could be combined 
across the studies and to ensure the generalizability of results to the largest proportion of 
young children. Studies were excluded if they were purely descriptive; were case studies; 
reported only demographic information; were qualitative studies; or included children 
with neurological or degenerative disorders, such as acquired immune deficiency syndrome 
(AIDS) or autism, or children who were blind or deaf The reason for these exclusions 
was to ensure the generalizability of combinable data that would be appropriate to the 
research questions. 

4. Languages studied had to include English and other alphabetic languages (i.e., languages in 
which the orthography or written symbol system represents speech sounds). This criterion 
ensured that the research findings would be maximally relevant and valid to English learning 
because English uses an alphabetic writing system. 

5. The study must include children between the ages of 0 and 5 or in kindergarten. Studies 
were included that targeted these ages alone, that included data on children from this age 
group that could be analyzed separately, or that included data on children from this age 
group that had been aggregated with the data of older children if the original research had 
shown that both groups performed the same. 

To ensure both comprehensiveness and representativeness of the research, additional searching 
was undertaken to identify studies that met those criteria but that may not have been identified 
in the electronic searches. This was accomplished by consulting nine previously published 
research reviews or meta-analyses to locate any papers cited in those reviews that were missed 
in these searches. Additionally, the panel searched the reference lists of all articles identified 
electronically that it had accepted, panelists nominated articles of which they were aware, and 
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panelists nominated a selected set of eight research journals, on the basis of relevance and quality, 
all issues of which (2001-2003) were hand searched for additional relevant articles (number of 
articles identified in each of these journals is noted in parentheses): 

The British Journal of Educational Psychology (no articles) 

Child Development (two articles) 

Developmental Psychology (no articles) 

Early Childhood Research Quarterly (no articles) 

Journal of Educational Psychology (three articles) 

Journal of Learning Disabilities (no articles) 

Reading Research Quarterly (one article) 

Scientific Studies of Reading (one article). 

Abstract and Article Review 

Project staff screened the 7,313 records, which included abstracts, using these selection criteria 
and categorized them according to their relevance to the four research questions. Ulrich’s 
International Periodicals Directory yfzs consulted to identify whether the journals were refereed. If 
staff were unable to determine whether a journal was refereed through this source, each journal 
editor was contacted and queried about that journals article-selection process. 

The results of this initial screening of citations produced a set of 1,824 studies from which panel 
members, in pairs, reviewed article abstracts to ascertain the study’s relevance to question 1. To 
be relevant to this research question, a study had to focus on a child skill or ability measured 
any time from birth through the beginning of kindergarten, and this skill or ability had to be 
statistically linked to a reading, writing, or spelling outcome measured from the conclusion of 
kindergarten to any time later, usually through a correlational procedure. If either of the two 
panelists determined that an article, as described in the title and abstract, was potentially relevant 
to the research question, it was added to the list of articles to be retrieved. On the basis of the 
abstract review, 685 full-text articles were retrieved for a second round of panel screening to 
determine whether the articles met the initial selection criteria and were relevant to question 
1 . Again, pairs of panelists reviewed articles independently. When the panelists’ ratings were 
in disagreement concerning an article’s status, they were contacted and asked to review their 
decisions and come to agreement about its inclusion. Occasionally, the two panelists would 
request a third panelist’s opinion about the disposition of a particular article. At any stage of the 
process, if an article was excluded, the reason for the rejection was documented. 

This process resulted in a final set of 275 studies identified for research question 1. Of these 275 
articles, 4l were rejected during the coding process for a variety of reasons (usually the data were 
reported inadequately and did not allow for coding of the needed information from the study), 
resulting in a final set of 234 articles for analysis. 

Coding Scheme and Coding Studies 

NELP identified seven categories for classifying study characteristics: report identification 
(ID number, citation, study coder, whether study rejected and reason); setting (program type, 
ages or grades, country of sample, population density); demographics (information about 
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subjects, languages, maternal education, family structure); research design (experimental, 
quasiexperimental, correlational); experimental design features (subject assignment, components 
of treatment, treatment fidelity); nonexperimental design features (correlational information, 
sample selection, measurement issues); and measurement information — all research designs (test 
names and categories, reliability, measures, means and standard deviations, ES). 

A coding instrument was developed using this coding scheme. A first reliability check on 
the coding instrument resulted in too many problems across coders to complete. To ensure 
consistency across coders, a coding manual was created with definitions of terms and operational 
definitions for coding procedures. Several panelists also made suggestions for revising the coding 
instrument. Using the revised coding instrument, a second coding trial was conducted with 
project coders. Results of an intercoder reliability check resulted in an agreement rate of 0.80. 
Instances of disagreement across coders were discussed and resolved. The coding manual was 
revised to reflect these coding problems. A final coding check involved a random selection of 
10 percent of the already-coded articles that were then sent to panel members for an accuracy 
check. About half of these articles were returned with comments, suggestions, or feedback. As 
the coding of articles continued, when coders were confronted with especially difficult-to-code 
articles, panelists were consulted to provide coding assistance. 

The panel decided to aggregate various measures used in the original studies into more general 
conceptual categories for the purposes of coding. For example, many articles were coded that 
measured aspects of reading that occurred in kindergarten or later. These measures were grouped 
into the larger constructs of (1) reading, including measures of individual word identification, 
decoding of nonsense words, and any other measure that tapped the accuracy of reading words; 

(2) reading comprehension, measures of students’ understanding of a written passage; and 

(3) reading fluency, which was represented by assessments that measure a student’s ability to 
accurately and quickly read a series of words or sentences. Many constructs were defined in this 
manner. (See Appendix l.B for the complete coding instrument.) Once studies were coded and 
ESs were compared, the panel classified outcome measures based on descriptions of variables, 
content of published measures, or a combination of those. A summary of the outcome domains 
into which variables were classified, including their definitions, is included in Appendix l.C. 

Missing Data 

As articles were coded, it became apparent that there were numerous instances of missing data, 
especially data that were critical to conducting a meta-analysis. In these instances, project 
staff attempted to contact article authors to retrieve the missing data. These attempts usually 
were not successful, as authors did not respond or no longer had the needed data. As a result, 
articles with missing data were excluded if the data that were missing were essential to carrying 
out the analyses. 

Data Entry and Cleaning 

After all the studies were identified and coded onto code sheets, they were entered into a database 
designed by the National Center for Family Eiteracy (NCFE). Two coders were in charge of 
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entering the data, and these coders double-entered approximately 1 0 percent of the information 
from the code sheets to allow an examination of the accuracy of the coding; the agreement 
was found to be above 0.95. Additionally, after the data were entered, two panelists reviewed 
approximately 20 percent of the original articles to ensure the accuracy of the data. Finally, any 
ES that was found to be an outlier (correlations below -0.30 or Cohen’s ({/values greater than 3.0 
or less than -3.0) was investigated for accuracy. 

Unit of Analysis 

Because many of the statistical procedures employed in a meta-analysis are sensitive to violations 
of the assumption of independence of observation, NELP decided to define the unit of analysis 
as the ESs obtained from independent groups, frequently, a single article represented one group. 
However, in some cases, a particular group of participants was used in more than one study 
(longitudinal studies with multiple assessments across time were the bulk of these studies). 

When this occurred, results were grouped from across these articles and treated as a single group. 
Additionally, some articles reported data from multiple independent groups. If the groups were 
defined as distinct from one another, they were treated as independent groups. 

Analysis 

Studies for research question 1 consisted primarily of correlational data. These studies often 
reported correlations of measures that the panelists considered to be measuring the same 
relationship (i.e., multiple measures of PA correlated with multiple measures of reading). When this 
occurred, the panel averaged the Eisher z-transformed correlations so that each study contributed 
only one ES per relationship. The formula used for converting the correlations into z-scores is 



Z 



0.5 



In 



1-f r 



1 — r 

V I y 



where 

Z = z-score that corresponds to the correlation from study i and 
r = correlation from study i. 



ESs obtained from each study were aggregated across studies by weighting the individual 
investigations by the inverse of the conditional variance of each z-transformed correlation: 



1 




where 

V. = variance of z-transformed correlation from study i and 
n. = number of subjects in study i. 
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And the weight (w. ) as 



1 

w. = — , 

V. 

I 

where 

V. = variance estimate for effect size from study i. 



Weighted average ESs were computed based on a fixed-effect model estimation procedure that 
weighted each ES by the inverse of the estimated variance {w^. These weighted mean Eisher 
z-correlations were then computed for each predictor-outcome relationship. The standard error 
(SE) of this estimate was computed by taking the square root of the inverse of the sum of the 
individual variance estimates that comprised that effect and was used to estimate 95 percent 
confidence intervals (CIs) for each Eisher z-transformed correlation. Einally, the estimate and the 
lower and upper bounds of the Cl were converted back into Pearson correlations. Additionally, an 
estimate of the homogeneity of the combined ESs ( Q} was estimated using the following formula: 









E 



IV. 



where 

k = number of studies contributing an effect, 
w. = weight of individual study i, and 
Z = z-transformed correlation from study i. 



Einally, the panel determined that, to report an ES, a minimum of three studies must have 
contributed to the estimation of that effect. 

In addition to the meta- analyses of simple correlations, the panel examined multivariate studies 
that were retrieved as part of the search for the first research question. Multivariate studies 
typically use multiple regression or similar analytic techniques to examine a variable s predictive 
utility in the context of other variables. These multivariate studies were coded for each of the 
primary predictor variables identified by the meta-analyses of simple correlations. Coding 
included a list of all additional variables included as predictors in the multivariate analyses 
and whether the primary predictor variable continued to account for unique variance in the 
decoding, reading comprehension, or spelling outcome measure. 



Methodology for Interventions 

Research question 2 asked about the effectiveness of instructional procedures, interventions, or 
other intentional actions that adults could take to improve the literacy development of young 
children (similar to the types of research questions that NRP raised). 
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Search Terms and Search Procedures 



The analyses for the first research question resulted in the identification of 1 3 measurable 
outcomes for early literacy instruction and intervention. Based on the findings from research 
question 1, project staff generated additional search terms using the 13 early literacy predictors 
(alphabet knowledge [AK] , language, concepts about print, environmental print, invented 
spelling, listening comprehension, name writing, PA, phonological short-term memory (STM), 
rapid automatic naming (RAN), verbal IQ, visual memory, visual perception) as superordinate 
(category) terms. Following search procedures similar to those used for research question 1, 
searches were conducted in both ERIC and PsycINFO databases. This search resulted in 974 
records. (See Appendix l.C for a complete list of search terms.) Table 1.2 shows the number of 
records generated by each superordinate term per search engine. 



Table 1.2. Records Generated per Search Engine for Predictor Variables 



Predictor Superordinate Term 


ERIC 


PsycINFO 


AK 


4 


14 


Language 


329 


403 


Concepts about print 


2 


3 


Environmental print 1 1 


Invented spelling 


2 


8 


Listening comprehension 


8 


11 


Name writing 


4 


10 


PA 


37 


54 


Phonological STM 1 1 


RAN 


0 


1 


Verbal IQ 


5 


16 


Visual memory 


0 


6 


Visual perception 


10 


43 



Abstract and Article Review 

Prior to conducting the search for interventions using the predictor variables as superordinate 
terms, panelists returned to the list resulting from the first screen of the original search. Rather 
than just looking for studies that linked a child skill or ability to reading outcomes through a 
correlational procedure, as was done for the first research question, it was necessary to review 
the abstracts to locate all studies of interventions that evaluated the impact of those interventions 
on reading, writing, or spelling measures or measures of any of the 13 predictor variables. 
Following the same procedures for abstract and article review for the first research question, this 
review resulted in the retrieval of 651 articles. Panel review of abstracts from the 974 records 
generated in the predictor search resulted in the retrieval of 280 articles. Combining the results 
of the previous search and the predictor search identified a total of 93 1 articles for retrieval and 
additional panel review. 

Following the same procedures as for review of articles for the first research question, the panel 
accepted a total of 136 articles for this second research question. In their review of articles for 
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this question, panelists used the following additional inclusion and exclusion criteria for the 

reasons given: 

1. The study must have employed an experimental or quasiexperimental design (QED). 

This criterion meant that an instructional procedures effectiveness or ability to cause an 
improvement in an early literacy skill was evaluated in the study. In QEDs, evidence must be 
presented to show that preexisting groups were equivalent before manipulation on variables 
thought to be related to the outcome. This would most generally take the form of an 
appropriate pretest. Experiments ensure the equivalence of initial performance by randomly 
assigning subjects to conditions. Quasiexperiments, because they evaluate instructional 
procedures’ effectiveness on the performance of intact groups, must use these measurement 
approaches to make sure that there are not existing differences among groups that could be 
misinterpreted as having resulted from the intervention. 

2. If the independent variable in the study was completely confounded by another grouping 
variable, it was rejected. Eor instance, if there are two classrooms and one is assigned to receive 
an intervention and the other to do business as usual, classroom is completely confounded 
with intervention; hence, any obtained effects may be due to the intervention or to something 
about the specific classroom or classroom teacher, and there is no way to disentangle these 
causal effects. This criterion ensures that outcomes attributed to an intervention were actually 
caused by that intervention and not some other confounding variable. 

3. One-group pretest-posttest-only designs were excluded. The most likely explanation of 
changes in skills from pre- to posttest in children is maturation. This criterion prevents the 
misattribution of changes that are due to maturation to an instructional procedure or effort. 

4. Studies that did not contain sufficient information to derive an ES were rejected. The 
results of these studies could not be statistically combined with the results of the other 
studies, so they were set aside. Again, no efforts were made to impute missing information, 
as explained earlier. 

5. Studies that did not have appropriate outcome measures were rejected. To determine whether 
an intervention had a positive impact on literacy learning, it was essential that the studies 
include appropriate outcome measures on which to evaluate the intervention’s effectiveness. 
Studies that used conventional measures of literacy (such as measures of decoding, reading 
comprehension, oral reading fluency, writing quality, spelling) could be included, as could 
studies that used any of the categories of measures identified in Chapter Three (e.g., 

oral language development, AK, cognitive ability). Although there were not searches for 
interventions aimed at improving children’s performance on readiness tests (tests that some 
schools use to predict later success in reading), if such measures were among those used 
in a study, their results are reported. Studies of children prior to or during kindergarten 
quite appropriately tend not to include such outcome measures. Accordingly, the panel 
set out to empirically determine which early skills were the strongest predictors of later 
literacy achievement (see Chapter Two). Studies that used outcome measures that evaluated 
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any of the important precursor skills identified in Chapter Two were also included in this 
analysis. (Chapters Three through Seven are organized by type of intervention rather than by 
particular outcome measure. This is consistent with NRP work and is more in accord with 
the types of decisions that must be made by those responsible for educational curricula.) 

6. Studies were excluded that were considered short-term trials of fewer than three weeks; 

focused on a particular stimuli or portion of a learning task; had relatively few subjects; and 
measures used were usually those limited to the study’s learning task, e.g., training children 
on letters and then measuring their learning of the specific letters reflected in the training 
materials. These kinds of studies can provide important clues to how learners process 
information or how instructional interventions might be formulated or improved. They do 
not provide a clear indication of the learning benefits of substantial or long-term efforts to 
improve learning, and, for this reason, these studies were not used toward answering the 
second research question. 

Categorizing Interventions 

To better understand the research conducted and to systematically code and analyze the studies, 
a subgroup of panelists and project staff categorized the 136 articles into the following five 
categories. It is important to note that the panel did not search for articles on these topical 
categories but instead searched for all studies of efforts to prepare young children for literacy 
success and derived the categories themselves from examining the articles that were identified in 
the search process. This contrasts with the NRP approach, which selected topics of study (e.g., 
phonemic awareness, phonics, technology, encouraging children to read, oral reading fluency, 
reading comprehension, teacher education, vocabulary) and then searched for interventions 
related to those topics. NRP necessarily took that approach to limit the scope of its investigation. 
Given the relatively limited number of intervention studies available to NELR the research team 
decided to examine all such studies rather than particular subsets. 

1 . Helping Children Make Sense of Print — Cracking the Alphabetic Code and Teaching 
Letters and Words 

2. Reading to and Sharing Books with Young Children 

3. Parent and Home Programs for Improving Young Children’s Literacy 

4. Preschool and Kindergarten Programs 

5. Language Enhancement. 

Further Checks for Article Inclusion 

Prior to finalizing the set of articles to be used to answer the second research question, the 
panelists implemented additional checks to ensure that all relevant studies that met the selection 
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criteria were identified and included in the synthesis. Panelists reviewed the reference lists of the 
136 included articles, reviewed all studies included in the research review completed by Halle 
et al. (2003), and sent a complete list of articles to an external group of 14 expert reviewers 
nominated by the panelists. The review of the article reference lists resulted in 130 potentially 
relevant citations not previously reviewed. Abstracts for these additional articles were examined, 
resulting in full-text review of another 65 articles. The review of the studies in Halle et al. 
resulted in one more accepted article. 

The review by the external experts identified four studies for review that had been missed, and 
two of these were accepted for inclusion. These items had been missed because of a discrepancy 
in the search terms across the PsycINFO and ERIC databases. As a result, the panel conducted 
additional searches of the PsycINFO databases using slightly revised age terminology. 

In addition to these checks, several researchers had notified panel members with inquiries about 
specific research that they thought should be included, often submitting a list of articles or the 
articles themselves. In these cases, article citations were checked against the panel’s lists, and 
individuals were notified about the status of an article’s inclusion. If articles had been rejected, 
explanations were provided detailing the reasons for exclusion. 

Revised Literature Searches 

The PsycINFO database was searched again, this time using age terms that better matched the 
index of that database. ERIC was not searched again, because there had been no such mismatch. 

The second search of PsycINFO repeated all the earlier search procedures and yielded the results 
reported in Tables 1.3 and 1.4. This identified 662 potential studies and, after the various 
screenings, expanded the total number of studies considered for question 1 to 299 and the 
number of studies considered for question 2 to 191. 



Table 1.3. Records Generated in Revised Original Search of PsycINFO 



Search Category 


Records 


Language 


109 


Cognition 


65 


Motivation 


65 


Schooling 


76 


Home and family 


60 


Word learning 


89 


Fluency 


13 


Reading comprehension 


92 


Miscellaneous 


93 
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Table 1.4. Records Generated in Revised Question 2 Search of PsycINFO 



Predictor Category 


Records 


AK 


5 


Language 


26 


Concepts about print 


0 


Environmental print 


0 


Invented spelling 


0 


Listening comprehension 


4 


Name writing 


2 


PA 


10 


Phonological STM 


0 


RAN 


0 


Verbal IQ 


3 


Visual memory 


2 


Visual perception 


6 


Literacy 


18 



Coding Scheme and Coding Studies 

The coding scheme for the second research question was similar to that of the first one, except 
that articles were further coded for group information from experimental and quasiexperimental 
studies. (See the coding instrument, form 5, for group information codes.) Specifically, the 
panel developed codes relevant to the five intervention categories (Appendix l.D) to aid in the 
investigation of any moderator effects that might be found for the various treatment effects. 

Unit of Analysis and Data Analyses 

The unit of analysis for the second research question was the ES obtained from independent 
groups as defined for the first research question. However, in this case, the ESs obtained were 
typically treatment-group contrasts with a control or comparison group, as compared with 
the within-group correlations used to answer the first research question. Effects from a single 
treatment-control comparison that were categorized as representing the same outcome construct 
were aggregated prior to analysis. ESs were compared using Cohen’s d: 




^ pooled 

where 

X t = mean of the treatment group at posttest, 
Xc = mean of the control group at posttest, and 
s , , = pooled standard deviation. 

pooled T 
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This </was corrected for small sample bias using the following formula: 



d' = 



1 - 



AN -9 



d. 



where 

N = total sample size ^treatment + control^ and 
d = Cohen's d. 



The SE associated with the adjusted ES estimate from each study is 



K+'V. I 

- '{ N,N ^2(n+n)’ 

where 

= sample size of the treatment group, 

= sample size of the control group, and 
d = adjusted Cohen's d. 



The weight (w) was computed as the inverse of the square of the SE. 

ESs were combined across independent groups assuming a random-effect model. Specifically, 
the homogeneity statistic Qwas computed (under the fixed-effect assumption) using the 
following formula: 




where 

k = number of studies contributing an effect, 
= weight of individual study, and 
d. = adjusted Cohen's d. 
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Using the Q statistic, the random-effect variance component was estimated using the following 
formula: 



V = 



Q-k-\ 



U“'. 



E 



w. 



E 



w. 



where 



Q = the homogeneity statistic, 

k = total number of effects that comprise that estimate, and 
w. = effect-size weight. 



This variance component was then added to the variance estimate for every ES, and the inverse 
weights {iv) were recalculated for each study. ESs across the studies were combined by weighting 
each effect by this new weight, and the SE of this estimate was computed by taking the square 
root of the inverse of the sum of the new individual variance estimates (with the random-effect 
variance component added in) that comprised that effect, finally, the panel determined that, 
to interpret an ES, a minimum of three studies must have contributed to the estimation of 
that effect. (ESs for single studies or combinations of two studies are included in some tables 
for information purposes or symmetry only; however, these statistics are neither analyzed nor 
discussed except to point out their inadequacy for interpretive purposes). 

If the homogeneity analysis ( Q statistic) showed that a particular set of ESs was heterogeneous 
(meaning that the likelihood that they all came from the same population of ESs was small), 
they were further analyzed using a set of predefined moderator variables that the panel developed 
and coded specific to each of the five categories of interventions. The moderator analyses 
were conducted using an analog to an analysis of variance (AN OVA) as described by Lipsey 
and Wilson (2001). Specifically, this technique partitions the total variability represented by 
the Q statistic into a between-subjects component that is attributable to a moderator and a 
within-subjects component that is considered residual variance, or variance unexplained by the 
moderator. If the between-subjects Qwas significantly different from zero for our categorical 
moderators, ES estimates were obtained for each level of the moderator, and SE and 95 percent 
CIs were computed for each one. We interpreted nonoverlapping CIs as being statistically 
significant from each other, even though we realize that this is only a close approximation to null- 
hypothesis significance testing that is typically performed on two or more means. Specifically, 
within an ANOVA framework, a pooled error term is used in the denominator when estimating 
statistical significance. By using CIs estimated for each mean, it is not taking advantage of 
the increased power and stability of a pooled estimator. Eor example, it should be noted that 
partially overlapping 95 percent CIs may actually be statistically significant at ^ < 0.05. This is 
because each Cl is based solely on the variance estimate (and SE) for that particular effect and 
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does not use a pooled estimate that is typically used when comparing two means. In the case 
of comparing two estimates, it will always be the case that, if the two CIs do not overlap, the 
difference between the two means will be significant zt p < 0.05. However, it is also possible 
that, when comparing two means that represent levels of a factor and this factor has three or 
more levels, two estimates found to have nonoverlapping 95 percent CIs would have been found 
not to be statistically different zt p> 0.05 using a pooled estimator. Specifically, if the two ES 
estimates being compared have smaller variance estimates than a pooled estimate across all levels 
of that factor, the confidence-interval approach would claim statistical significance, while the 
approach using the pooled estimator would not. In general, however, the use of CIs as a means of 
discussing significant effects is a conservative, although admittedly less powerful, approach. 

Effect Sizes 

ESs are useful because they are standardized and can be compared across studies. In this report, 
the strength of an ES is characterized as small if it ranges up to 0.30 (meaning as big as 20 
percent of a standard deviation), moderate if it is in the 0.50-0.79 range, and large (meaning a 
difference at least half a standard deviation or larger) if it is over 0.80 (Cohen, 1988). 

ESs are a function of not only the experimental intervention, but the instruction being provided 
to the comparison children as well. True control groups are rare in education, and individual 
teachers and parents vary a great deal in their teaching and interaction routines. The intervention 
studies examined here contrast performance in a variety of comparison situations. In some 
cases, the researchers structure an alternative treatment for the comparison students, usually to 
enhance the contrast. In far more cases, the researcher accepts the naturally occurring variation 
in existing classrooms and households and intervenes by enhancing some aspect of support in 
the experimental situations. This means that the comparison groups may be providing some 
version of the intervention, albeit without the intensifying help of the researcher. For example, 
in studies of book sharing, the researcher might provide particular books for the experimental 
classrooms or households, might teach teachers and parents particular ways to share books with 
children, or might impose a specific schedule to increase the amount of book sharing that takes 
place. The comparison teachers and parents, left to their own devices, might also be reading to 
children during the study, though one suspects that they would not do it as much or as well as 
the experimental teachers and parents. ESs tend to be larger when interventions are compared 
with true controls and smaller when they are compared with alternate treatments. 



Methodology for Mediators and Moderators 

If, for a particular intervention and outcome variable, there is a significant average effect across 
a collection of studies, it is worthwhile to see what can be deduced about the nature of that 
effect. Toward that end, homogeneity analysis is used to determine whether the variation in 
individual effects are just normal sampling error or whether they might be the result of how the 
various research studies were conducted, differences in features of the intervention, or differences 
in the children themselves. If the homogeneity analysis indicates that the individual study 
effects are from different distributions, further analysis was warranted to try to find patterns of 
differences in effects across these studies. All studies were coded with regard to the participants. 
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interventions, contexts, and research variables, and differences in these variables were explored 
to help explain the variation in ESs (to try to understand why different versions of similar 
interventions differ in their degree of effectiveness). 

In addition to this standard coding of studies, panelists worked in subgroups relevant to the 
subcategories of interventions. These subgroups of panelists identified additional comparisons 
unique to the set of studies in each category to determine whether variations in the interventions 
were more or less successful under various conditions. 
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Appendix 1.A: Original Search Categories and Terms 

Search Categories 

1. Language. This set identifies < 2 // articles dealing with language, language abilities, 

language development, and language learning. These are all synonyms and related terms. 
They should be linked together with or in a search — meaning that we will identify a set 
of all articles that focuses on any one of the following topics: 
child language 
dialect 

distinctive features (language) 

expressive language 

grammar 

intonation 

language 

language acquisition 
language development 
language fluency 
language impairments 
language learning 
language processing 
language skills 
language typology 
lexical development 
lexicology 

listening comprehension 

metalinguistics 

morphology 

oral language 

phonemic 

phonemic awareness 
phonetic 

phonological awareness 

phonological processing 

phonological sensitivity 

phonology 

pragmatics 

psycholinguistics 

receptive language 

semantics 

semiotics 

speech 

speech communication 

speech skills 

syntax 

verbal communication 
verbal development 
vocabulary 
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2. Cognition. This set identifies all articles dealing with cognitive abilities (excluding 
language issues), including learning, perception, memory, and intellect. These are all 
synonyms and related terms. They should be linked together with or in a search — 
meaning that we will identify a set of all articles that focuses on any one of the 
following topics: 
aptitude 
attention 
attention control 
attention span 
auditory perception 
automatic processing 
automaticity 
cognition 
cognitive ability 
cognitive behavior 
cognitive development 
cognitive flexibility 
cognitive functioning 
cognitive load 
cognitive models 
cognitive processes 
cognitive psychology 
cognitive research 
cognitive skills 
cognitive strategies 
cognitive structures 
cognitive style 
concept development 
concept formation 
conceptual change 
conceptual tempo 
encoding 

information processing 
intelligence 

IQ 

learning processes 
long-term memory 
memorization 
memory 
metacognition 
perception 
rapid naming 
recall 

recognition 
retention 
schema 
schema theory 
schemata 
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short-term memory 
social cognition 
visual perception 

3. Motivation. This set identifies all terms that are related to motivation, interest, attitude, 
and affective factors in learning. These are all synonyms and related terms. They should 
be linked together with or in a search — meaning that we will identify a set of all articles 
that focuses on any one of the following topics: 

affective domain 
aspiration 
attitude 
curiosity 

external motivation 

fear of success 

goal orientation 

incentives 

intention 

interest 

interests 

internal motivation 
learning motivation 
motivation 

motivation techniques 
praise 

reading attitudes 

reading interests 

reading motivation 

rewards 

satisfaction 

self motivation 

social desirability effects 

success 

4. Schooling. This set identifies any influences on early literacy by any kind of schooling 
or care arrangement or instructional approach or program. These are all synonyms and 
related terms. They should be linked together with or in a search — meaning that we will 
identify a set of all articles that focuses on any one of the following topics: 

ancillary school services 
child care 
child caregivers 
day-care centers 
day-care effects 
early childhood education 
early experience 
early identification 
early intervention 
Even Start 
family day care 
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family literacy 
Head Start 
home schooling 
individualized reading 
initial teaching alphabet 
Kenan model 

language experience approach 
parents as teachers 
prekindergarten classes 
prekindergarten teachers 
preschool clinics 
preschool curriculum 
preschool experience 
preschool programs 
preschool teachers 
reciprocal teaching 
special education 
sustained silent reading 

5. Home and family. This set identifies any environmental influences in the home. 

These are all synonyms and related terms. They should be linked together with or in a 
search — meaning that we will identify a set of all articles that focuses on any one of the 
following topics: 
brothers 

caregiver interaction 
family 

family (sociological unit) 
family environment 
family influence 
family life 
family literacy 
family problems 
family relationships 
fathers 
grandparents 
home experiences 
lower-class parents 
middle-class parents 
mothers 

parent aspiration 
parent attitude 
parent background 
parent-child relationship 
parent education 
parent influence 
parent participation 
parent-school relations 
parental attitudes 
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parenthood education 

parenting 

parenting skills 

parents 

siblings 

sisters 

6. Word learning. This set identifies all information about the learning of words and 
word parts in reading and writing. Anything dealing with decoding the printed word 
or encoding (spelling) is included here. They should be linked together with or in a 
search — meaning that we will identify a set of all articles that focuses on any one of the 
following topics: 

alphabets 

basic vocabulary 

consonants 

context clues 

decoding 

grapheme 

invented spelling 

letters (alphabet) 

morphemes 

morphophonemic 

orthographic symbols 

pattern recognition 

phoneme-grapheme correspondence 

phonemes 

phonemic awareness 

phonics 

phonology 

rhyming 

sight method 

sight vocabulary 

spelling 

structural analysis 

syllables 

vowels 

word lists 

word recognition 

word study skills 

7. Fluency. This set is to identify all information about the learning of fluency (speed, 
accuracy, expression) in reading. Anything dealing with fluency in oral and silent reading 
is included here. They should be linked together with or in a search — meaning that we 
will identify a set of all articles that focuses on any one of the following topics: 

context clues 
eye voice span 
fluency 

inner speech (subvocal) 
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intonation 
miscue analysis 
oral interpretation 
oral reading 
prosody 

reading aloud to others 
reading rate 
silent reading 

8. Reading comprehension. This set identifies all information about the learning of fluency 
(speed, accuracy, expression) in reading. Anything dealing with fluency in oral and silent 
reading is included here. They should be linked together with or in a search — meaning 
that we will identify a set of all articles that focuses on any one of the following topics: 

comprehension 
content-area reading 
critical reading 
reader response 
reader-text relation 
schema theory 
story grammar 
text structure 

9. Miscellaneous. This set identifies all information about reading and writing that is not 
included in the other sets (including writing and concepts of print). They should be 
linked together with or in a search — meaning that we will identify a set of all articles that 
focuses on any one of the following topics: 

concept of word 
concepts of print 
conventions of print 
developmental delays 
environmental print 
directionality 
prevention 
reading readiness 
special-need students 
story reading 
reading habits 
reading process 
reading strategies 
writing (composition) 
writing ability 
writing achievement 
writing attitudes 
writing contexts 
writing development 
writing difficulties 
writing evaluation 
writing improvement 
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writing instruction 
writing motivation 
writing processes 
writing readiness 
writing research 
writing skills 
writing strategies 

10. Age group. This set identifies children by age or grade level. We want to find anything 
written on children from birth to age 5, including preschool and kindergarten. They 
should be linked together with or in a search — meaning that we will identify a set of all 
articles that focuses on any one of the following topics: 

early childhood education 
early experience 
infants 

kindergarten children 
preschool children 
toddlers 
young children 

11. Literacy. This set identifies any articles that deal with reading and writing. They should 
be linked together with or in a search — meaning that we will identify a set of all articles 
that focuses on any one of the following topics: 

language arts 
literacy 

beginning reading 
content-area reading 
corrective reading 
critical reading 
early reading 
functional reading 
independent reading 
oral reading 
recreational reading 
remedial reading 
silent reading 
story reading 
reading ability 
reading achievement 
reading comprehension 
decoding 
reading diagnosis 
reading difficulties 
reading failure 
reading improvement 
reading instruction 
reading motivation 
reading processes 
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reading programs 
reading readiness 
reading research 
reading skills 
reading strategies 
reading-writing relationship 
writing (composition) 
writing ability 
writing achievement 
writing attitudes 
writing contexts 
writing development 
writing difficulties 
writing evaluation 
writing improvement 
writing instruction 
writing motivation 
writing processes 
writing readiness 
writing research 
writing skills 
writing strategies 

Searches 

1 . Role of language in early reading development 

Pool 1 : Link all items in set 1 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 

2. Role of cognitive development in early reading development 

Pool 1 : Link all items in set 2 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 

3. Role of motivational factors in early reading development 

Pool 1 : Link all items in set 3 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 

4. Role of schooling or educational effort in early reading development 

Pool 1 : Link all items in set 4 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 
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5. Role of home and family on early reading development 
Pool 1 : Link all items in set 5 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 

6. Role of word knowledge (decoding) in early reading development 
Pool 1 : Link all items in set 6 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 

7. Role of fluency in early reading development 
Pool 1 : Link all items in set 7 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 

8. Role of comprehension in early reading development 
Pool 1 : Link all items in set 8 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 

9. Role of miscellany in early reading development 
Pool 1 : Link all items in set 9 by or. 

Pool 2: Link all items in set 10 by or. 

Pool 3: Link all items in set 1 1 by or. 

Link pools 1,2, and 3. 



Appendix 1.B: NELP Coding Instrument 

Formi: Report Identification 

Article ID No: 

1.1. Article ID No.: 

1.2. Citation 

Author(s): 

Title: 

Source: 

Year: 
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Volume (Issue): 



Page Numbers: 

Abstract: (copy/paste from abstract list) 

1.3. Coder ID 

Coder 1: 

Coder 2: 

1.4. Report Rejected: /'C/rec/r/ / or 

□ 1 . Accepted 

□ 2. Rejected 

If rejected, give reason: 

1 .5. Research Question Relevance: (Check all that apply) 

□ 1.RQ1 

□ 2. RQ2 

□ 3. RQ3 

□ 4. RQ4 

Form 2: Setting 
Article ID No. 

2.1 . Country(ies) sample drawn from: (check# / or 2) 

□ I.U.S. 

□ 2. Other(s): 

2.2. Population Density (Check all that apply) 

□ Urban 

□ Suburban 

□ Rural 

□ Mixed 

□ Unknown 

2.3. Grade Level(s)/Ages at Entry (Check all that apply) 

□ Infants (up to 12 months) 

□ Toddlers (12 to 24 months) 

□ Two-year-olds 

□ Three-year-olds 

□ Four-year-olds 

□ Five-year-olds (not in Kindergarten) 

□ K 

□ Mixed 

□ Other 



DEVELOPING EARLY LITERACY: REPORT OP THE NATIONAL EARLY LITERACY PANEL 



27 



2.4. Grade Level(s)/Ages throughout study (Check all that apply) 

□ Infants (up to 12 months) 

□ Toddlers (12 to 24 months) 

□ Two-year-olds 

□ Three-year-olds 

□ Four-year-olds 

□ Five-year-olds (not in Kindergarten) 

□ K 

□ 1 
□ 2 

□ 3 

□ Mixed 

□ Other: 

2.5. Program Type (Check all that apply) 

□ Preschool 

□ Public 

□ Private 

□ Unknown 

□ Flead Start 

□ Child Care 

□ Even Start 

□ Kindergarten 

□ Other: 

□ None 

2.6. Program Setting (Check all that apply) 

□ Flome-based 

□ Center-based 

□ School-based 

□ Unknown 

2.7. Duration of Program (Check all that apply) 

□ Flalfday 

□ Full day 

□ Unknown 

2.8. Flome Setting (check # 1 or 2) 

□ 1 . Yes 

□ 2. No 
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Form 3: Demographics 



Article ID No. 

3.1 . Number of groups with demographic information: 

(If more than one, name the groups, e.g. Treatment, Control): 
Enter the group names below 



. Demographic Total of Group 1 Group 2 Group 3 Group 4 

Information All Groups 

3.2 Number of Subjects 
(beginning of study) 

3.3 Number of Subjects 
(end of study) 



3.4 


Boys 


Number: 


Number: 


Number: 


Number: 


Number: 




(Do the Math) 


Percent: 


Percent: 


Percent: 


Percent: 


Percent: 














3.5 


Girls 


Number: 


Number: 


Number: 


Number: 


Number: 




(Do the Math) 


Percent: 


Percent: 


Percent: 


Percent: 


Percent: 



3.6 Mean age 
in MONTHS 
(beginning) 



3.7 Age range 
in MONTHS 
(beginning) 



3.8 


SES 


(place a check next to 


(place a check next to 


(place a check next to 


(place a check next to 


(place a check next to 






all that apply) 


all that apply) 


all that apply) 


all that apply) 


all that apply) 






□ Low 


□ Low 


□ Low 


□ Low 


□ Low 






□ Other 


□ Other 


□ Other 


□ Other 


□ Other 






□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 


3.9 


Ethnicity/Race 
(Give percent, if 
known, and do 
the math) 


(place a check 
next to all that 
apply and give 
percent of each, 
if known) 


(place a check 
next to all that 
apply and give 
percent of each, 
if known) 


(place a check 
next to all that 
apply and give 
percent of each, 
if known) 


(place a check 
next to all that 
apply and give 
percent of each, 
if known) 


(place a check 
next to all that 
apply and give 
percent of each, 
if known) 






□ Lat/Hisp 


□ Lat/Hisp 


□ Lat/Hisp 


□ Lat/Hisp 


□ Lat/Hisp 






□ AfrAm 


□ AfrAm 


□ AfrAm 


□ AfrAm 


□ AfrAm 






□ Cauc 


□ Cauc 


□ Cauc 


□ Cauc 


□ Cauc 






□ Other 


□ Other 


□ Other 


□ Other 


□ Other 






□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 
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Item # 



Demographic Total of Group 1 Group 2 Group 3 Group 4 

Information All Groups 



3.10 Language(s) 


(Check either i1 
or 2 for each or 
i3 if unknown) 


(Check either 11 
or 2 for each or 
#3 if unknown) 


(Check either a 
or 2 for each or 
i3 if unknown) 


(Check either i1 
or 2 for each or 
i3 if unknown) 


(Check either #1 
or 2 for each or 
#3 If unknown) 




Home: 

□ 1 . English 

□ 2. Other: 


Home: 

□ 1 . English 

□ 2. Other: 


Home: 

□ 1. English 

□ 2. Other: 


Home: 

□ 1 . English 

□ 2. Other: 


Home: 

□ 1 . English 

□ 2. Other: 




□ 3. Unknown 


□ 3. Unknown 


□ 3. Unknown 


□ 3. Unknown 


□ 3. Unknown 




Instruction: 

□ 1 . English 

□ 2. Other: 


Instruction: 

□ 1 . English 

□ 2. Other: 


Instruction: 

□ 1. English 

□ 2. Other: 


Instruction: 

□ 1 . English 

□ 2. Other: 


Instruction: 

□ 1 . English 

□ 2. Other: 




□ 3. Unknown 


□ 3. Unknown 


□ 3. Unknown 


□ 3. Unknown 


□ 3. Unknown 


3.11 Family Structure 


piace a check next 
to aii that appiy 


place a check next 
to all that apply 


place a check next 
to all that apply 


place a check next 
to all that apply 


place a check next 
to all that apply 




□ Single Parent 

□ Two Parent 

□ Teen Parent 

□ Other 


□ Single Parent 

□ Two Parent 

□ Teen Parent 

□ Other 


□ Single Parent 

□ Two Parent 

□ Teen Parent 

□ Other 


□ Single Parent 

□ Two Parent 

□ Teen Parent 

□ Other 


□ Single Parent 

□ Two Parent 

□ Teen Parent 

□ Other 




□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 



3.12 Maternal 


(place a check next 


(place a check next 


(place a check next 


(place a check next 


(place a check next 


Education 


to all that apply 


to all that apply 


to all that apply 


to all that apply 


to all that apply 


(Give percent. 


and give percent, 


and give percent, 


and give percent, 


and give percent. 


and give percent, 


If known): 


if known): 


if known): 


if known): 


if known): 


if known, and 
do the math) 


□ Less than high 


□ Less than high 


□ Less than high 


□ Less than high 


□ Less than high 




school: 


school: 


school: 


school: 


school: 




□ Some high 


□ Some high 


□ Some high 


□ Some high 


□ Some high 




school: 


school: 


school: 


school: 


school: 




□ Eligh school 


□ Eligh school 


□ Eligh school 


□ Eligh school 


□ Eligh school 




diploma: 


diploma: 


diploma: 


diploma: 


diploma: 




□ GED: 


□ GED: 


□ GED: 


□ GED: 


□ GED: 




□ Vocational: 


□ Vocational: 


□ Vocational: 


□ Vocational: 


□ Vocational: 




□ Some 


□ Some 


□ Some 


□ Some 


□ Some 




colleae: 


colleae: 


colleae: 


colleae: 


colleae: 




□ College 


□ College 


□ College 


□ College 


□ College 




araduate: 


araduate: 


araduate: 


araduate: 


araduate: 




□ Other: 


□ Other: 


□ Other: 


□ Other: 


□ Other: 




□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 


□ Unknown 
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Item # 



Item* 

Information 


Total of 
All Groups 


Group 1 


Group 2 


Group 3 


Group 4 












3.13 Child 

Characteristics 


(place a check next 
to all that apply and 
specify the Items 
marked, if known): 


(place a check next 
to all that apply and 
specify the items 
marked, if known): 


(place a check next 
to all that apply and 
specify the Items 
marked, if known): 


(place a check next 
to all that apply and 
specify the Items 
marked, if known): 


(place a check next 
to all that apply and 
specify the items 
marked, if known): 




□ Perinatal/ 

Neonatal 

Conditions: 


□ Perinatal/ 

Neonatal 

Conditions: 


□ Perinatal/ 

Neonatal 

Conditions: 


□ Perinatal/ 

Neonatal 

Conditions: 


□ Perinatal/ 

Neonatal 

Conditions: 




□ ESL 


□ ESL 


□ ESL 


□ ESL 


□ ESL 




□ Psychopathology: 


□ Psychopathology: 


□ Psychopathology: 


□ Psychopathology: 


□ Psychopathology: 




□ Premature birth 


□ Premature birth 


□ Premature birth 


□ Premature birth 


□ Premature birth 




□ Developmental 
Delay: 


□ Developmental 
Delay: 


□ Developmental 
Delay: 


□ Developmental 
Delay: 


□ Developmental 
Delay: 




□ Other: 


□ Other: 


□ Other: 


□ Other: 


□ Other: 




□ Unspecified 


□ Unspecified 


□ Unspecified 


□ Unspecified 


□ Unspecified 


3.14 Parent 

Characteristics 


(place a check next to 
all that apply and 
specify the items 
marked, if known): 


(place a check next to 
all that apply and 
specify the items 
marked. If known): 


(place a check next to 
all that apply and 
specify the items 
marked, if known): 


(place a check next to 
all that apply and 
specify the items 
marked, if known): 


(place a check next to 
all that apply and 
specify the items 
marked, if known): 




□ History of 
Reading 
Problems: 


□ History of 
Reading 
Problems: 


□ History of 
Reading 
Problems: 


□ History of 
Reading 
Problems: 


□ History of 
Reading 
Problems: 




□ ESL 


□ ESL 


□ ESL 


□ ESL 


□ ESL 




□ Psychopathology: 


□ Psychopathology: 


□ Psychopathology: 


□ Psychopathology: 


□ Psychopathology: 




□ Income 


□ Income 


□ Income 


□ Income 


□ Income 




□ Parental 
Education: 


□ Parental 
Education: 


□ Parental 
Education: 


□ Parental 
Education: 


□ Parental 
Education: 




□ Other: 


□ Other: 


□ Other: 


□ Other: 


□ Other: 




□ Unspecified 


□ Unspecified 


□ Unspecified 


□ Unspecified 


□ Unspecified 



3.15 Other 

Demographics 
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Form 4: Research Design 



Article ID No. 

4. 1 . Research Type (Check one of the following) 

□ 1 . Experimental Designs — Interventions Complete Forms 5 & 7 

□ 2. Experimental Designs — Other Complete Forms 5 & 7 

□ 3. Correlational Designs Complete Forms 6 & 8 

□ 4. Intervention study with correlations Complete Forms 5, 6, & 7 

□ 5. Qualitative Design Discontinue 

□ 6. Other group designs with correlations Complete Forms 5, 6, & 7 

Form 5: Experimental Designs — Interventions 

Article ID No. 

5.1. Group Design Study 

□ Experimental (Check all that apply) 

□ Control group 

□ No control group 

□ Matched 

□ Not matched 

□ Quasi-experimental (Check all that apply) 

□ Control group 

□ No control group 

□ Matched 

□ Not matched 

5.1 .1 . What Type of Controls? (check one of the following) 

□ 1 . Pre/Post 

□ 2. Flistorical 

□ 3. No Treatment Controls 

□ 4. Other 

5.1 .2. Is the study design flawed or confounded, if so what is the reason (check one of the following) 

□ 1. Fatal Flaw 

□ 2. Confounded 

□ 3. Other: 

5.2. Measurement Design (check one of the following) 

□ 1 . Pre-test and Post-test 

□ 2. Post-test only 

□ 3. Pre-test/Post-test/Follow-up 

□ 4. Other: 
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5.3. Number of treatment groups: 

5.4. Number of treatments: 

5.5. Number of control groups: 

5.6. Selection restrictions (e.g., low or high reading score(s)): 



5.7. Description of the intervention(s) or control group instruction(s): 

Group 1 Group 2 Group 3 Group 4 



5.8. Level of intervention (place a check next to all that apply) 

□ Individual 

□ Small Group 

□ Full Class (large group) 



5.9. Implementers — for each group (place a check next to all that apply) 

Group 1 Group 2 Group 3 Group 4 



□ Evaluator/researcher 

□ Other researchers 

□ Practitioners/Teachers 

□ Parents 

□ Volunteers 

□ Technology 

□ Other 



□ Evaluator/researcher 

□ Other researchers 

□ Practitioners/Teachers 

□ Parents 

□ Volunteers 

□ Technology 

□ Other 



□ Evaluator/researcher 

□ Other researchers 

□ Practitioners/Teachers 

□ Parents 

□ Volunteers 

□ Technology 

□ Other 



□ Evaluator/researcher 

□ Other researchers 

□ Practitioners/Teachers 

□ Parents 

□ Volunteers 

□ Technology 

□ Other 



5.9.1 . Intervention Settings — for each group (place a check next to all that apply) 

Group 1 Group 2 Group 3 Group 4 



□ Classroom 

□ Pull out of Classroom 

□ University Lab 

□ Elome 

□ Other 



□ Classroom 

□ Pull out of Classroom 

□ University Lab 

□ Elome 

□ Other 



□ Classroom 

□ Pull out of Classroom 

□ University Lab 

□ Elome 

□ Other 



□ Classroom 

□ Pull out of Classroom 

□ University Lab 

□ Elome 

□ Other 
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5.10. Duration of Intervention (for each group, if given) 



Group 1 Group 2 Group 3 



Number of weeks by 
individual/group 

Number of sessions 
per week by 
individual/group 

Number of minutes 
per session by 
individual or group 

Total number 
of sessions by 
individual/group 

Unknown or Not 
Applicable (i.e. 
regular classroom 
instruction) 



5.10.1 Length of Long-term Intervention (check one) 

□ 1. Less than an academic year 

□ 2. Academic year 

□ 3. Whole calendar year 

□ 4. More than 1 2 months but less than 24 months 

□ 5. More than 2 years 

□ 6. Other 

5.11. Attrition: 

Was there attrition of subjects? (check one of the foilowing) 

□ 1 . Yes 

□ 2. No 

□ 3. Unknown 

Was there differential loss of subjects across groups? (check one of the foilowing) 

□ 1 . Yes 

□ 2. No 

□ 3. Unknown 

Did retained subjects differ from lost subjects? (check one of the following) 

□ 1 . Yes. If yes, the difference: 

□ 2. No 

□ 3. Unknown 



Group 4 
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5.12. Comparison Group Rating (for control group(s) only) (Check all that apply for each group) 



Group 1 



Group 2 



Group 3 



Group 4 



□ Active Control 

□ Alternative Intervention 

□ Initial Group 
Equivalency 

□ Control change agents 

□ Low attrition at post 

□ Doesn't apply/Not a 
control group 

□ Other 



□ Active Control 

□ Alternative Intervention 

□ Initial Group 
Equivalency 

□ Control change agents 

□ Low attrition at post 

□ Doesn't apply/Not a 
control group 

□ Other 



□ Active Control 

□ Alternative Intervention 

□ Initial Group 
Equivalency 

□ Control change agents 

□ Low attrition at post 

□ Doesn't apply/Not a 
control group 

□ Other 



□ Active Control 

□ Alternative Intervention 

□ Initial Group 
Equivalency 

□ Control change agents 

□ Low attrition at post 

□ Doesn't apply/Not a 
control group 

□ Other 



5.1 3. Durability of Effects (Check one of the following: Place a 0 In any empty fields, for example: _0_ days _0 

weeks _0_ months _3_ years) 

□ 1. Immediate posttesting 

□ 2. Post testing days weeks months years 

5.14. Amount of training and type of support for implementors: 



5.15. Characteristics of training (Check all that apply) 

□ Use of manual 

□ Supervision/consultation 

□ Audio/videotapes 

□ Informal or formal training sessions 

□ Other: 

□ Unknown 

5.1 6. Eidelity of Implementation (Check all that apply) 

□ Ongoing supervision/consultation and observations 

□ Audiotapes 

□ Videotapes 

□ Use of manual 

□ None 

□ Unknown 

□ Other: 
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5.17. What is the Intervention Focus (Outcome)? (Check all that apply) 

□ Oral language (general vs. specific) 

□ Phonological Awareness (what level: rhyme, syllable, phoneme, letter sounds based on audition) 

□ Print Awareness (letter names, letter sounds (based on print), or other like CAP) 

□ Comprehensive Literacy (i.e., no specific target) 

□ General Stimulation (projects like Abecedarian) 

□ ESL 

□ Classroom Environment Enhancement for Literacy 

□ Other 

5.17.1 What is the Intervention Focus (Material/Style)? (Check all that apply) 

□ Materials only 

□ Systematic Instructional Strategy 

□ Direct Instruction (not directed by the child) 

□ Other 

5.17.2 Who is the Intervention Intended to Target (Age)? (Check all that apply) 

□ Toddlers (i.e., children not in preschool, <3 years) 

□ Preschooler (3 to 5-year-olds) 

□ Kindergarten 

□ Other 

5.17.3 Who is the Intervention Intended to Target (Special Populations)? (Check all that apply) 

□ Global 

□ At-Risk only 

□ Not at-risk only 

□ SLL 

□ Other 
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Form 6: Non-Experimental Designs — Correlational 

Article ID No. 

6.1 . Restriction Range 

Any reason to think that sample selection factors would restrict performance ranges or variability? (check 
either #/ or 2 and provide an expianation if#1 is checked) 

□ 1 . Yes 

□ 2. No 

6.2. Measurement Issues 

Any important alterations to the data for the purposes of analysis? (check either #1 or 2 and provide an 
expianation if /I is checked) 

□ 1 . Yes 

□ 2. No 
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Form 6: Non-Experimental Designs — Correlations & Regressions 



Article ID No. 



Table 6.3 Zero-order correlations between preschool predictor variables and literacy outcomes 



Article 

ID# 



Preschool Age/Grade 

Predictor PPV Category when 

Variahle measured 



Literacy Literacy 

Dutcome Category 



Article ID No. 



Table 6.4 Semi-Partial correlations between preschool predictor variables and literacy outcomes 



Article 

ID# 



Variahles 

Controlled 



Preschool 

Predictor 

Variahle 



PPV 

Category 



Age/Grade 

when 

measured 



Literacy 

Dutcome 



□ IQ(non-Verbal) 

□ Other 



□ IQ(non-Verbal) 

□ Other 



□ IQ(non-Verbal) 

□ Other 

□ lO(non-Verbal) 

□ Other 
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Age/Grade 

when 


Correlation 


P value 


N 

(number of 


measured 






subjects) 



Literacy 

Category 



Age/Grade 

when 

measured 



R2 



P 

value 



N 

(number of 
subjects) 
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Form 7: Measurement Information — Intervention Designs 



Article ID No. 



Table 7.1 Measurement Information - Intervention Designs 



Test 


Test Test Reliability 


Test Type 


Groups Pretest 


Pre-test 


Pretest 


Post- 


Post- 


Adjusted 


Name 


Category 


(check one 
for each test) 


Mean 


SD 


N for 
each 
group 


test 

Mean 


test 

SD 


Post-test 

Mean 

(only If pretest 
Is covariate) 



□ 1. □ 1 . Published 



□ 2. Cited 
reference 
in text 



□ 2. Unpublished 

□ 3. Unknown 



□ 3. NA 



1 

2 

3 



4 



□ 1. □!. Published 



□ 2. Cited 
reference 
in text 



□ 2. Unpublished 

□ 3. Unknown 



□ 3. NA 



1 

2 

3 



4 
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Posttest 


Time 


Follow- 


Follow- 


Follow- 


Time 


Follow- 


Follow- 


Follow- 


Time 


Pre/ Post 


Effect 


N for 


since 


up 1 


up 1 SD 


up 1 N 


since 


up 2 


up 


up 2 


since 


Interaction 


Size 


each 


Pretest 


Mean 




for each 


Posttest 


Mean 


2SD 


N for 


Follow- 


p value 




group 


(months) 






group 


(months! 






each 


up 1 







group (months! 
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Appendix 1.C: Outcome and Predictor Variables 



Table I.C.1. Variables and Descriptions 



Variable 


Description 


AK 


Knowledge of letter names or letter sounds, measured with recognition or naming 
test. Typically assessed with measure developed by investigator. 


Arithmetic 


Ability to perform mathematical operations, such as addition, subtraction, or 
counting, and knowledge of numbers. Assessed with measure developed by 
investigator or, more commonly, with standardized tests, such as the Peabody 
Individual Achievement Test, mathematics subtest, or Woodcock-Johnson Tests 
of Achievement®, applied problems subtest. 


Concept knowledge 


Knowledge of general concepts, such as colors, comparatives, directions, materials, 
positions, quantities, relationships, sequences, shapes, sizes, social and emotional 
states, characteristics, textures and time. Measured with a standardized test, such 
as the Bracken Basic Concept Scale. 


Concepts about print 


Knowledge of print conventions (e.g., left-right, front-back) and concepts (book 
cover, author, text). Assessed with either measure developed by investigator or 
using measure, such as Clay's Concepts About Print Test. 


Decoding nonwords 


Use of symbol-sound relations to verbalize pronounceable nonwords (e.g., "gleap," 
"taip"). Typically measured with a standardized measure, such as the Word Attack 
subtest of the Woodcock Reading Mastery Test. 


Decoding not otherwise 
specified (NOS) 


Use of symbol-sound relations to verbalize real words, pronounceable nonwords, 
or both. Sometimes assessed using a measure developed by investigator, and 
sometimes a combination of two decoding tests, such as both Word Attack and 
Word Identification from the Woodcock Reading Mastery Test. Often insufficient 
information provided to determine exact nature of decoding task. 


Decoding words 


Use of symbol-sound relations to verbalize real words or use of orthographic 
knowledge to verbalize sight words (e.g., "have," "give," "knight"). Typically 
assessed with a standardized measure, such as the Word Identification subtest of 
the Woodcock Reading Mastery Test. 


Environmental print 


Ability to identify product or company name for common product or establishment 
(e.g., "Coke," "McDonalds"). 


Invented spelling 


Ability to use sound-symbol relations but not necessarily orthographic rules to write 
words (e.g., "BK" for bike, "RM" for arm). 


IQ 


Scores from full-scale intelligence measures, such as the Wechsler Preschool and 
Primary Scales of Intelligence™ or Stanford-Binet Intelligence Scale. 
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Variable 


Description 


Oral language 


Ability to produce, comprehend, or both aspects of spoken language, including 
semantics, syntax, or both. Often measured by a standardized test, such as 
the Peabody Picture Vocabulary Test or the Clinical Evaluation of Language 
Fundamentals®. 


Performance IQ 


Scores from nonverbal subtests or subscales from intelligence measures, such 
as the Wechsler Preschool and Primary Scales of Intelligence or Stanford-Binet 
Intelligence Scale. 


PA 


Ability to detect, manipulate, or analyze components of spoken words independent 
of meaning. Examples include detection of common onsets between words 
(alliteration detection) or common rime units (rhyme detection): combining syllables, 
onset rimes, or phonemes to form words; deleting sounds from words; counting 
syllables or phonemes in words; or reversing phonemes in words. Often assessed 
with a measure developed by the investigator, but sometimes assessed with a 
standardized test, such as the Comprehensive Test of Phonological Processing. 


Phonological NOS 


Phonological task with insufficient information provided to determine whether PA, 
phonological memory, or a combination. 


Phonological STM 


Ability to remember spoken information for a short period of time. Typical 
tasks include digit span, sentence repetition, and nonword repetition from both 
investigator-created measures and standardized tests. 


Print awareness 


Tasks combining elements of AK, concepts about print, and protodecoding 
(beginning or early decoding). 


RAN letters or digits 


Rapid naming of sequentially repeating random sets of letters, digits, or both. Often 
assessed with investigator-created measure. 


RAN objects or colors 


Rapid naming of sequentially repeating random sets of pictures of objects (e.g., 
"car," "tree," "house," "man") or colors. Often assessed with investigator-created 
measure. 


Readiness 


Composite measure including combinations of aspects of AK, concepts of print, 
vocabulary, memory, and PA. Often insufficient information provided to determine 
exact content of measure. 


Reading comprehension 


Measures of comprehension of meaning of written language passages. Typically 
measured with standardized test, such as the Passage Comprehension subtest 
of the Woodcock Reading Mastery Test. 


Reading NOS 


Measure of reading with insufficient information provided to determine whether 
decoding, comprehension, or both was assessed. 


Spelling 


Ability to use sound-symbol relations and orthographic rules to write words using 
conventional spelling. 
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Variable 


Description 


Visual memory 


Short-term recall of visually presented information (e.g., recall of object or shape 
presented to child). 


Visual motor 


Ability to copy or draw figure or shape from a model. 


Visual perception 


Ability to match or discriminate visually presented symbols. 


Writing or writing name 


Ability to write letters in isolation on request or write own name. Often assessed 
with measure created by examiner. 



Appendix 1.D: Intervention Search Categories and Terms 

1. Predictor categories and terms 

Alphabet Knowledge (predictor subcategory) 

alphabets or (letter identification) or letters or alphabet 

Language (predictor subcategory) 

(child language) or dialect or (distinctive features) or (expressive language) or grammar or 
language or (language acquisition) or (language development) or (language fluency) or (language 
impairments) or (language learning) or (language processing) or (language skills) or (language 
typology) or (lexical development) or lexicology or (listening comprehension) or metalinguistics 
or morphology or (oral language) or psycholinguistics or (receptive language) or semantics or 
syntax or (verbal communication) or (verbal development) or vocabulary or sociolinguistics or 
(second language acquisition) or (second language development) or bilingualism 

Concepts About Print (predictor subcategory) 

(concepts about print) or (concepts of print) or (concept of word) or directionality or 
(conventions of print) or (print awareness) 

Environmental Print (predictor subcategory) 

(environmental print) or (environmental text) 

Invented Spelling (predictor subcategory) 

(invented spelling) or (developmental spelling) or (emergent spelling) 

Listening Comprehension (predictor subcategory) 

(listening comprehension) or (aural learning) or (listening comprehension tests) or 
(verbal comprehension) 
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Name Writing 

(name writing) or (emergent writing skills) or writing or (early writing) 

Phonological Awareness (predictor subcategory) 

(phonological awareness) or phonemic or (phonemic awareness) or phonetic or (phonological 
processing) or (phonological sensitivity) or phonology or phoneme or phonological or 
rhyming or rhymes or blending or segmenting or (sound categorization) or (sound isolation) 
or (sound awareness) or syllables or vowels 

Phonological STM (predictor subcategory) 

(phonological short-term memory) or (phonological memory) or (memory for sentences) or 
(digit span) 

RAN (graphological and nongraphological) (predictor subcategory) 

(rapid naming) 

Verbal IQ (predictor subcategory) 

(verbal intelligence) or (verbal ability) or (verbal IQ) 

Visual Memory (predictor subcategory) 

(visual memory) or (visuospatial memory) or (spatial memory) 

Visual Perception (predictor subcategory) 

(visual perceptual) or (visual perception) or (word perception) or (form perception) or 
(binocular vision) or (eye fixation) or (stereoscopic vision) or (visual discrimination) or (visual 
tracking) or (visual spatial ability) or (visual spatial memory) 

2. Intervention Category and Terms 

Interventions (ERIC) 

(group design) or (control group) or (treatment group) or (experimental design) or 
intervention or (education experiments) or (early intervention) or pretests or posttests or 
(program effectiveness) or (educational improvement) or (experimental groups) or (matched 
groups) or (quasi-experimental design) 

Interventions (PsycINFO) 

(group design) or (control group) or (treatment group) or (experimental design) or 
intervention or (early intervention) or (treatment effectiveness evaluation) or (between group 
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designs) or experimentation or (experimental subjects) or pretesting or (repeated measures) 
or posttesting or (program evaluation) or (educational program evaluation) or (experimental 
replication) 

3. Age Category and Terms 

Age category and terms (original search) 

(early childhood education) or (early experience) or infants or (kindergarten children) or 
(preschool children) or toddlers or (young children) or (infant development) 

Age category and terms (revised search) 

(early childhood education) or (early experience) or infants or (kindergarten children) or 
(preschool children) or toddlers or (young children) or (infant development) or (kindergarten 
students) or (preschool students) 

4. Literacy Category and Terms 

literacy or (beginning reading) or (content area reading) or (corrective reading) or (critical 
reading) or (early reading) or (functional reading) or (independent reading) or (oral reading) 
or (recreational reading) or (remedial reading) or (silent reading) or (story reading) or (reading 
ability) or (reading achievement) or (reading comprehension) or decoding or (reading 
diagnosis) or (reading difficulties) or (reading failure) or (reading improvement) or (reading 
processes) or (reading research) or (reading skills) or (reading strategies) or (reading writing 
relationship) or writing or composition or (writing ability) or (writing achievement) or 
(writing attitudes) or (writing contexts) or (writing development) or (writing difficulties) or 
(writing evaluation) or (writing improvement) or (writing processes) or (writing readiness) 
or (writing research) or (writing skills) or (writing strategies) or dyslexia or fluency or (inner 
speech) or subvocal or (orthographic knowledge) or prosody or reading or (reading assessment) 
or (reading disabilities) or (reading disability) or (reading evaluation) or (reading rate) or (sight 
method) or (sight vocabulary) or (sight words) or spelling or (word automaticity) or (word 
learning) or (word recognition) or writing or (writing assessment) 
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Appendix 1.E: Intervention Subcategory Codes 

Subcategory 1; Making Sense of Print 

1 . Phonological Awareness Training (check all that apply) 

□ Sub-phonemic (syllable, rhyme, onset, rime) 

□ Phonemic (phonemes) 

□ Analysis (deletion, elision, counting) 

□ Synthesis (blending) 

□ With alphabet knowledge training (letter sounds) 

□ With alphabet knowledge (letter names) 

2. Alphabet Knowledge Training (check all that apply) 

□ Letter sounds 

□ Letter names 

□ Other: 

3. Child s Reading Ability (check all that apply) 

□ Knows no letter names/sounds 

□ Knows some letter names/sounds Specify: 

□ Non-reader 

□ Reader Specify level: 

Subcategory 2; Shared Reading 

Background 

1 . Paternal Education 

G Less than high school 
G Some high school 
G High school diploma 
GGED 
G Vocational 
G Some college 
G College graduate 
G Other 
G Unknown 

2. Number of Adults in the Home 

G Number: 

G Unknown 

3. Number of Children in the Home 

G Number: 

G Unknown 

4. Parent Reading Ability 

G Specify: 

G Unknown 
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Parent/Teacher Training and Support 

1. Who is providing book sharing/reading? (Check all that apply) 

□ Parent 

□ Teacher 

□ Computer 

□ Other: 

□ Unknown 

2. Amount of training provided 

□ Minutes: 

□ Other: 

□ Unknown 

3. Books provided (check all that apply) 

□ No books provided 

□ Given to family all at one time 

□ Given to family as the program proceeded 

□ Given to classroom or school 

□ Lending library program 

□ Other: 

□ Unknown 

4. Number of books provided 

□ None 

□ Number: 

□ Unknown 

5. Kinds of books provided (check all that apply) 

□ Storybooks 

□ Informational books 

□ Alphabet books 

□ Other: 

□ Unknown 

6. Characteristics of Training (check all that apply) 

□ Print focus (guiding children to look at print, asking questions about letters, 
pointing at words) 

□ Content focus (asking questions about the story, comments about the story, 
etc.) 

□ Parents/teachers trained to ask questions Specify type: 

□ Parents/teachers trained to elicit responses from or actions by the child 

□ Parents/teachers trained how to reply/ react to child responses 

□ Parents/teachers trained to direct child’s attention to the pictures 

□ Parents/teachers encouraged to read books repeatedly to their child 

□ Parents/teachers trained to elicit retellings of the story 

□ Other: 

□ Unknown 
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7. Who trained the parents (check all that apply) 

□ Teacher 

□ Librarian 

□ Other parents 

□ Researcher 

□ Other: 

□ Unknown 

8. Training materials (check all that apply) 

□ Manual/Handbook 

□ Video 

□ Demonstration 

□ Guided practice with own children 

□ Other: 

□ Unknown 

Parent/Child Interaction 

1 . Interaction Characteristics (check all that apply) 

□ Child could see print being read 

□ Parent draw child’s attention to print Specify: 

□ Child engaged during the reading (looking at book, pointing, book-related 
comments, turning pages, etc.) 

□ Child answering parent questions 

□ Parent answering child questions 

□ Other: 

□ Unknown 

2. Number of times book read per sitting 

□ Number: 

□ Unknown 

3. Number of books read per sitting 

□ Number: 

□ Unknown 

4. Number of times parent read to child 

□ Number: 

□ Unknown 

5. Length of book reading 

□ Minutes: 

□ Other: 

□ Unknown 

Subcategory 3; Parent and Home Programs 

1 . Intervention Focus (check all that apply) 

□ Oral language only 
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□ Oral and written language 

□ General Development 

□ Affective Development 

□ Other: 

□ Unknown 

2. Family Focus 

□ Family (general) Child 

□ Specific Parent Child 

□ Parent and Child 

□ Other: 

□ Unknown 

3. Intervention Characteristics (check all that apply) 

□ Mother-Child Dyads 

□ Home-based; scripted, highly structured 

□ International 

□ Other: 

□ Unknown 

4. Materials 

□ None 

□ Books 

□ Toys 

□ Other: 

□ Unknown 

Subcategory 4; Preschool and Kindergarten Experience 

1 . Type of Experience (check all that apply) 

□ Promotion/Retention 

□ Head Start 

□ Head Start + Abecedarian 

□ Head Start + Darcee 

□ Head Start + Follow Through 

□ Other Early Childhood: 

□ None 

□ Preschool + Head Start 

□ Preschool + Abecedarian 

□ Preschool + Social Services 

□ Preschool + Nutrition 

□ Preschool + Low Ratios 

□ Preschool + In-service professional development 

□ Preschool + Parent Involvement 

□ Preschool + Other: 

□ Unknown 
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2. Years of Experience (check all that apply) 

□ Preschool Specify: 

□ Kindergarten Specify: 

□ Other: 

□ Unknown 

3. Program Length 

□ Full Day Number: 

□ Half Day: Number: 

□ Other: 

□ Unknown 

4. Support Services Provided (check all that apply) 

□ None 

□ Transition classes/services 

□ Parent Workshops 

□ Family Education 

□ Home Visitation 

□ Other: 

□ Unknown 

Subcategory 5: Language Enhancement 

1 . Language Impairment (check all that apply) 

□ Expressive delay 

□ Grammar 

□ Phonology 

□ Receptive/Listening comprehension 

□ Severe 

□ Other: 

□ None 

□ Unknown 

2. Who delivered instruction (check all that apply) 

□ Parent 

□ Clinician 

□ Teacher 

□ Other: 

□ Unknown 

3. Media Provided 

□ None 

□ Computer 

□ Television 

□ Unknown 
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4. Type of Instruction (check all that apply) 

□ Programmed Instruction (e.g., Distar, Hanen Program for Parents, Language 
for Learning, etc.) 

□ Direct Language Production 

□ Play-focused 

□ Teacher-directed 

□ Child-initiated 

□ Other: 

□ Unknown 

5. Focus of Instruction (check all that apply) 

□ Consonants 

□ Pragmatics 

□ Semantics 

□ Syntax 

□ Vowel sounds 

□ Words 

□ Other: 

□ Unknown 

6. Types of strategies (check all that apply) 

□ Expansion 

□ Focused stimulation 

□ Implicit correction 

□ Interactive dialogue 

□ Modeling 

□ Modeling + reinforcement 

□ Operant conditioning 

□ Parallel talk 

□ Recasting 

□ Repetition 

□ Scaffolding 

□ Shared reference 
n Songs/rhymes 

□ Vocal imitation 

□ Other: 

□ Unknown 

7. Materials 

□ None 

□ Toys 

□ Books 

□ Other: 

□ Unknown 
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The objective of convening the National Early Literacy Panel (NELP) was to identify interventions 
and practices that promote positive outcomes in literacy for preschool children. Prior to the 
onset of formal instruction in kindergarten and beyond, few interventions that directly target the 
conventional literacy skills (decoding, reading comprehension, spelling, writing) can be found. 

Not only are such interventions few, but it is unlikely that such interventions would be considered 
developmentally appropriate for preschool-age children. Additionally, the panel suspected that there 
would be few studies aimed at enhancing later conventional literacy skills that would follow children 
for a sufficient length of time to observe such effects. Consequently, the panel determined that 
the first step in its research synthesis would need to involve the identification of skills that strongly 
predicted later conventional literacy skills. This determination produced the panel’s first research 
question described in Chapter One: What are the skills and abilities of young children (age birth 
through five years or kindergarten) that predict later reading, writing, or spelling outcomes? 

Although there have been a plethora of theoretical writings, professional opinions, and best- 
practice documents proposing skills that should be considered the precursors to conventional 
literacy, to date, there has been little systematic empirical summation of research demonstrating 
that these early literacy skills predict later conventional literacy' Two recent documents 



^ The National Center for Education Statistics has had an important series of longitudinal investigations under way, the Early 
Childhood Longitudinal Studies (ECLS), since the late 1990s. These studies have been tracking childrens learning and 
development from kindergarten and birth on to identify growth trajectories in school learning and the relationship of various 
child and environmental variables to these trajectories. Unfortunately, none of these investigations had directly examined the 
predictability of later literacy performance on the basis of early childhood variables. Although reports are now emerging that are 
making this link, these were not available when the NELP work was undertaken. The reports that have emerged from this project 
so far have not looked at the relationship of individual child performance variables in relationship to later literacy success, but they 
have examined composites of variables, and these analyses have generally been consistent with the findings reported here (e.g., 
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provided consensus or narrative summaries of a portion of the research literature concerning the 
relationship between early precursor skills and later conventional literacy skills. Snow, Burns, and 
Griffin (1998), in their report of the National Research Council’s panel on preventing reading 
difficulties in young children, identified weaknesses in oral language, phonological awareness 
(PA), and alphabet knowledge (AK) as prime targets of intervention to prevent the occurrence 
of significant reading problems. Similarly, Whitehurst and Lonigan (1998) identified skills 
in the domains of oral language, print and letter knowledge, and phonological processing as 
encompassing two aspects (outside-in and inside-out skills) of emergent literacy that are related 
to later conventional forms of reading and writing. Whereas these two documents provided 
the beginnings of a structure to understand those skills that may serve as the developmental 
precursors to reading and writing abilities, neither document was based on a comprehensive 
summary of the published literature. 

As summarized in the description of NELP methods, a list of search terms in nine categories was 
created (language, cognition, motivation, schooling, home and family, word learning, fluency, 
reading comprehension, miscellaneous). Systematic electronic searches were conducted for 
published articles that were indexed by these search terms. These searches yielded 7,313 articles. 
An iterative process of review of these articles for relevance (i.e., initial screening, abstract review, 
article review) resulted in the elimination of 7,038 articles. Most articles were eliminated because 
they did not include usable data (e.g., no correlations reported, qualitative study, theoretical 
article), did not include relevant variables (e.g., no reading- or writing- related outcome variable), 
or did not include the age group that was the focus of this analysis (e.g., children of kindergarten 
age or younger). Of the 275 articles that passed all three tiers of the screening process, 4l were 
rejected during the coding process because they did not report sufficient information to allow 
coding of effects. A revised literature search conducted after the panel determined that age 
terms had prevented the identification of key resources in PsycINFO resulted in a final total 
of 299 articles. These articles reported the results of studies that involved the measurement of 
one or more child skills assessed when children were between birth and five years of age or in 
kindergarten and the measurement of one or more child outcomes on a conventional literacy skill 
assessed when children were in kindergarten or older. 

Included in the panel’s discussion of identification of early literacy skills linked to later outcomes 
in conventional literacy were specification of conditions required for something to be considered 
an early literacy skill and specification of the domain of conventional literacy skills. In defining 
an emergent or early literacy skill, the panel required that two conditions be met. First, the skill 
needed to be present before the conventional literacy skill, and second, the skill needed to be 
related to or predictive of a conventional literacy skill. These conditions establish two of the 
three criteria required for causal interpretation: temporal precedence and covariation. Given the 
correlational nature of the data in the articles included in this meta-analysis, the third condition 
for causal interpretation (ruling out of alternative explanations for the observed covariation 
between variables) could not be established. However, as described later (see the Multivariate 

composites including measures of alphabet knowledge, phonological awareness, visual-motor skills, and concepts of print have 
collectively been predictive of later reading achievement), but it would be worthwhile to have a more specific comparative analysis 
of these meta-analytical results with the longitudinal results of those large data sets. Some of the recent studies to emerge using 
ECLS data to examine literacy development include Chatterji (2006), Kaplan and Walpole (2005), Son and Meisels (2006), and 
Denton and West (2002). 
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Studies section), the panel attempted to provide some clarification of the covariation between 
identified variables. The panel defined conventional literacy skill in the receptive and expressive 
domains, which provided a symmetric classification for decoding or encoding print. Receptive 
skills included the ability to decode print, which included such outcomes as decoding words, 
decoding nonwords, and decoding fluency and measures of reading comprehension. Expressive 
skills included spelling and composition. The articles retrieved included outcomes in all of these 
domains except for composition. Hence, the resulting meta- analysis provides results for two 
receptive conventional literacy skills (decoding and reading comprehension) and one expressive 
conventional literacy skill (spelling). 

The primary analyses for this part of the research synthesis addressed the identification of 
preschool and kindergarten predictors of conventional literacy skills. Several sets of secondary 
analyses were conducted to answer finer-grained questions concerning factors related to the 
observed strength of the association between these predictors and conventional literacy skills. 
These secondary analyses included questions concerning the age of assessment for the predictor 
variable and the age of assessment for the measurement of the conventional literacy outcome 
variable. Additionally, where possible, the panel’s investigation examined the differential strength 
of association for different facets or means of assessment of the predictor variables. 

Primary Analyses 

Prior to conducting the meta-analysis, predictor and outcome variables were grouped according 
to the construct measured by using either the identified standard test of the construct (e.g., 
standardized tests of decoding, such as the Woodcock Reading Mastery Test) or the description 
of the measure included in the method section of the articles (e.g., a measure described as 
requiring children to delete a sound from a word spoken by the examiner was classified as a PA 
measure). The computation of an effect size (ES) for a predictor variable required that there be 
a minimum of three studies that contributed a correlation to the analysis. Therefore, variables 
indexing potential early literacy skills that have not been included in at least three studies are not 
represented. Eor these analyses, average correlations that were 0.50 or larger (i.e., the predictor 
variable explains at least 25 percent of the variance in the outcome variable) were designated 
as strong relationships-, average correlations that were between 0.30 and 0.49 (i.e., the predictor 
variable explains between 9 percent and 25 percent of the variance in the outcome variable) 
were designated as moderate relationships-, and average correlations that were below 0.30 (i.e., 
the predictor variable explains less than 9 percent of the variance in the outcome variable) were 
designated as weak relationships. 

The results for the primary analyses include information about the relationship between a 
predictor variable and a conventional literacy outcome presented in tabular form. Eor each 
predictor variable, the information in the table includes the average correlation across all studies, 
the numbers of studies on which the correlations were based, the numbers of children tested for 
each correlation, the 95 percent confidence interval (Cl) for the average correlation (i.e., -95 
percent Cl, +95 percent Cl), and the Q statistic for the average effect. 

No statistic can be measured with absolute accuracy. All measures are imperfect, and the 
collection of data from samples that are smaller than the total population never provides more 
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than an estimate of the population results. For that reason, CIs are a useful tool for estimating the 
accuracy of an estimate. A Cl provides a range of possible values within which a population value 
may lie, with a given confidence. For example, say the average correlation across 1 0 small studies 
was 0.50 and the Cl for this estimate ranged from 0.40 to 0.60. It is unlikely that the 0.50 
correlation obtained for these samples is exactly what would be found if the entire population 
of all young children were to be tested. But the Cl indicates that there is a 95 percent chance 
that this population value falls somewhere within this CL CIs are useful for showing whether 
a statistic is statistically significant (if it does not include zero in the interval, it is statistically 
significant), and they can be used to compare correlations across variables, with the caveats noted 
in the methodology chapter. That is, for pairs of variables in which the Cl does not overlap, the 
average correlations are significantly different from each other at the p < 0.05 level. 



The Q statistic provides a metric of the heterogeneity of observed effects (correlations) across 
the studies that contributed a value to the average correlation. A large and statistically significant 
Q statistic indicates that the observed correlations were unlikely to be sampled from the same 
population of correlations (i.e., the observed correlations have a high degree of heterogeneity). 
For instance, a significant Q statistic would result if the correlations included in the average were 
drawn from two populations of children whose average correlations, if computed separately, 
would be significantly different from each other. One example of such a situation would be 
if the average correlation included effects from both three- and five-year-old children and the 
average correlation between a predictor variable measured when children were three years of 
age and the outcome were significantly higher than the average correlation between a predictor 
variable measured when children were five years of age and the outcome. Another example 
would be if multiple measurement methods were used in the studies that contributed to the 
average correlation when the method of measurement of a predictor variable or an outcome 
variable influenced the size of the correlation. It is important to note that a significant Q statistic 
identifies the presence of significant heterogeneity in the distribution of observed effects but does 
not identify the cause of that heterogeneity. 



It is also important to note other factors that can affect the size of the obtained average 
correlations. Another factor that can affect the size of the correlation is the length of time from 
the assessment of the predictor to the measurement of the dependent variable. Correlations 
would presumably be lower, on average, with longer intervals of time in between assessments. 

The reliability of the measures being used also will affect the size of the correlations, as will 
restrictions of range resulting from the assessment of groups of children who are drawn from a 
narrow range of a distribution (such as would occur with the testing of only learning disabled 
students) or if there were floor or ceiling effects on the measures used to estimate the correlations. 



All of these factors can affect the size of the estimated correlations used in these analyses, and 
this must be acknowledged as a limitation of the results. However, it should also be noted that, 
for any of these factors to influence the relative ordering of the predictors by size of correlation, 
these factors would have to be operating differentially for the different predictors. That is, for 
the relative ordering of the predictors by size of correlation to be affected, there would have to be 
systematic differences in reliability, restriction of range, or length of time between assessments. 



58 



DEVELOPING EARLY LITERACY: REPORT OF THE NATIONAL EARLY LITERACY PANEL 




What Skills Measured in the Early Childhood Period or in Kindergarten Were Related 
to Decoding? 

The overall results for the predictive relations between variables measured in kindergarten or 
earlier and children’s decoding skills are presented in Tables 2.1 and 2.1a. Identical information 
is provided in two formats in these two tables. In Table 2.1, the predictors are listed in order 
of the strength of correlations with later decoding measures; in Table 2.1a, these same data are 
reorganized conceptually so that the various groups of predictors (e.g., rapid automatic naming 
[RAN] ; phonology) can be more easily compared. 



Table 2. 1. Average Correlations for Prediction of Decoding by Variables Measured in Kindergarten or 
Earlier (organized by size of correlation) 



Predictor Variable 


Avg. f 


95% Cl 


N 

Studies 


N 

Children 


a 






Lower 

Bound 


Upper 

Bound 








Decoding nonwords 


0.72 


0.68 


0.75 


8 


763 


132.74" 


Spelling 


0.60 


0.56 


0.63 


7 


1,184 


25.13" 


Invented spelling 


0.58 


0.53 


0.62 


10 


778 


39.03" 


Reading NOS 


0.57 


0.54 


0.60 


3 


1,739 


1.59 


Decoding NOS 


0.53 


0.48 


0.57 


5 


877 


59.85" 


Decoding words 


0.52 


0.50 


0.55 


21 


4,121 


396.32" 


Reading comprehension 


0.52 


0.47 


0.58 


5 


700 


132.14" 


AK 


0.50 


0.48 


0.52 


52 


7,570 


719.94" 


Readiness 


0.50 


0.46 


0.53 


5 


1,988 


28.20" 


Writing or writing name 


0.49 


0.45 


0.53 


10 


1,650 


25.18" 


Arithmetic 


0.45 


0.43 


0.48 


14 


3,929 


184.80" 


10 


0.45 


0.41 


0.48 


13 


2,015 


55.34" 


PA 


0.40 


0.39 


0.42 


69 


8,443 


505.84" 


RAN letters and digits 


0.40 


0.36 


0.43 


12 


2,081 


40.27" 


Concepts about print 


0.34 


0.31 


0.37 


12 


2,604 


75.13" 


Oral language 


0.33 


0.31 


0.34 


63 


9,358 


248.58" 


RAN objects and colors 


0.32 


0.29 


0.35 


16 


3,100 


25.37* 


Phonological NOS 


0.31 


0.17 


0.44 


3 


174 


0.85 


Performance 10 


0.30 


0.27 


0.34 


15 


2,792 


25.18* 


Print awareness 


0.29 


0.22 


0.35 


6 


683 


120.14" 


Environmental print 


0.28 


0.22 


0.34 


6 


1,042 


60.84" 


Phonological STM 


0.26 


0.24 


0.29 


33 


4,863 


163.85" 


Visual motor 


0.25 


0.20 


0.30 


14 


1,316 


24.96* 


Visual memory 


0.22 


0.17 


0.26 


8 


1,708 


12.11 


Visual perception 


0.22 


0.18 


0.26 


16 


2,551 


151.75" 



*=p<0.05: **p<o.o/. 
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Table 2. la. Average Correlations for Prediction of Decoding by Variables Measured in Kindergarten or 
Earlier (predictor variables grouped conceptually) 



Predictor Variable 


Avg. f 


95% Cl 




N 

Studies 


N 

Children 


a 






Lower 


Upper 












Bound 


Bound 








AK 


0.50 


0.48 


0.52 


52 


7,570 


719.94** 


Concepts about print 


0.34 


0.31 


0.37 


12 


2,604 


75.13** 


Environmental print 


0.28 


0.22 


0.34 


6 


1,042 


60.84** 


Print awareness 


0.29 


0.22 


0.35 


6 


683 


120.14** 


Decoding nonwords 


0.72 


0.68 


0.75 


8 


763 


132.74** 


Decoding NOS 


0.53 


0.48 


0.57 


5 


877 


59.85** 


Decoding words 


0.52 


0.50 


0.55 


21 


4,121 


396.32** 


IQ 


0.45 


0.41 


0.48 


13 


2,015 


55.34** 


Performance IQ 


0.30 


0.27 


0.34 


15 


2,792 


25.18* 


Arithmetic 


0.45 


0.43 


0.48 


14 


3,929 


184.80** 


Oral language 


0.33 


0.31 


0.34 


63 


9,358 


248.58** 


PA 


0.40 


0.39 


0.42 


69 


8,443 


505.84** 


Phonological NOS 


0.31 


0.17 


0.44 


3 


174 


0.85 


Phonological STM 


0.26 


0.24 


0.29 


33 


4,863 


163.85** 


RAN letters and digits 


0.40 


0.36 


0.43 


12 


2,081 


40.27** 


RAN objects and colors 


0.32 


0.29 


0.35 


16 


3,100 


25.37* 


Readiness 


0.50 


0.46 


0.53 


5 


1,988 


28.20** 


Reading NOS 


0.57 


0.54 


0.60 


3 


1,739 


1.59 


Reading comprehension 


0.52 


0.47 


0.58 


5 


700 


132.14** 


Spelling 


0.60 


0.56 


0.63 


7 


1,184 


25.13** 


Invented spelling 


0.58 


0.53 


0.62 


10 


778 


39.03** 


Visual motor 


0.25 


0.20 


0.30 


14 


1,316 


24.96* 


Visual memory 


0.22 


0.17 


0.26 


8 


1,708 


12.11 


Visual perception 


0.22 


0.18 


0.26 


16 


2,551 


151.75** 


Writing or writing name 


0.49 


0.45 


0.53 


10 


1,650 


25.18** 



*=p<o.05:**=p<o.oi 



The predictor variables that were most highly correlated with decoding were other conventional 
literacy variables.^ That is, in those studies in which a conventional literacy skill (decoding, 
reading comprehension, spelling) was measured when children were in kindergarten or earlier, 
these skills were substantially related to measures of decoding that were obtained when children 
were in kindergarten or later. All of these effects could be classified as strong relationships. 

Several variables typically thought of as representing early literacy development yielded average 
correlations that could be classified as moderate to strong relationships. Childrens AK yielded 
a strong relationship of 0.50 averaged across 52 studies involving 7,570 children. Measures 
of children’s ability to write or write their names resulted in a moderate relationship of 0.49 
averaged across 10 studies involving 1,650 children. Children’s PA skills yielded a moderate 



^ The NOS classification (not otherwise specified) is used in cases in which the exact nature of the variable cannot be determined. 
For instance, Decoding NOS in Table 1 . 1 in Chapter One refers to situations in which the measure of predictor variable could be 
identified as decoding but whether it involved the decoding of words or nonwords could not be determined. 
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relationship of 0.40 averaged across 69 studies involving 8,443 children. Measures of oral 
language yielded a moderate relationship of 0.33 in 63 studies involving 9,358 children. Both 
types of rapid naming measures also yielded moderate relationships. For rapid naming of letters 
or digits, an average correlation of 0.40 from 12 studies involving 2,081 children was obtained, 
and, for rapid naming of objects or colors, an average correlation of 0.32 from 16 studies 
involving 3,100 children was obtained. Finally, measures of children’s concepts about print 
yielded a moderate relationship of 0.34 in 12 studies involving 2,604 children. 

Additional variables that yielded at least moderate relationships included phonological not 
otherwise specified (NOS) (average r= 0.31 in three studies), arithmetic (average r = 0.45 in 
14 studies), IQ (average r= 0.45 in 13 studies), performance or nonverbal IQ (average r = 0.30 
in 15 studies), and readiness (average r= 0.50 in five studies). Measures of readiness contain a 
mix of early literacy constructs; hence, although a moderate relationship was obtained, it is not 
clear which component skill or skills contributed to this outcome. Similarly, the phonological 
NOS variable was not specified clearly enough to determine the nature of the skill being assessed 
that led to this moderate relationship. Measures of arithmetic knowledge, IQ, and performance 
IQ seem likely to index general cognitive abilities that are not specific to conventional literacy 
outcomes. All other variables yielded weak relationships with decoding, including print 
awareness, environmental print, phonological short-term memory (STM), and all assessment 
measures involving visual skills. 

Examination of the CIs for the correlations (see Table 2.1) suggested that both AK and “writing 
or writing” name yielded significantly higher correlations with decoding than did other early 
literacy variables, such as PA, concepts about print, oral language, and both types of rapid 
naming tasks. Similarly, PA was a stronger predictor of decoding than were concepts about 
print and oral language. Overall, oral language was the weakest predictor of decoding among 
the predictor variables yielding at least a moderate relationship. In fact, oral language was not 
a significantly stronger predictor than many of the predictor variables that yielded only weak 
relationships. Finally, the Q statistics indicated that, for most predictor variables, there was 
significant heterogeneity in the observed sample of correlations. 

What Skills Measured in the Early Childhood Period or in Kindergarten Were Related to 
Reading Comprehension? 

The overall results for the predictive relationships between variables measured in kindergarten or 
earlier and children’s reading comprehension skills are presented in Tables 2.2 and 2.2a (again, 
the second table provides the same information organized conceptually by type of variable). As 
seen in Table 2.2a, fewer studies examined reading comprehension than examined decoding. Two 
predictor variables yielded correlations that represented strong relationships. Readiness measures 
yielded an average correlation of 0.59 in three studies involving 348 children, and measures 
of concepts about print yielded an average correlation of 0.54 in three studies involving 535 
children. As noted in the preceding section, the readiness measures contain a mix of skills and 
do not provide information about which skill or skills contribute to the observed relationships. 
Additionally, the correlations for these two variables were not significantly higher than the 
variables that obtained large correlations but were classified as moderate relationships. 
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Table 2.2. Average Correlations for Prediction of Reading Comprehension by Variables Measured in 
Kindergarten or Earlier (organized by size of correlation) 



Predictor Variable 


Avg. r 




95% Cl 


N 

Studies 


N 

Children 


a 






Lower 

Bound 


Upper 

Bound 








Readiness 


0.59 


0.51 


0.65 


3 


348 


11.76" 


Concepts about print 


0.54 


0.48 


0.60 


3 


535 


3.31 


AK 


0.48 


0.45 


0.51 


17 


2,038 


28.36* 


Print awareness 


0.48 


0.39 


0.56 


4 


347 


8.54* 


PA 


0.44 


0.41 


0.48 


20 


2,461 


58.08" 


RAN ietters and digits 


0.43 


0.34 


0.52 


3 


333 


1.46 


RAN objects and coiors 


0.42 


0.38 


0.47 


6 


1,146 


6.78 


Decoding nonwords 


0.41 


0.30 


0.50 


3 


282 


72.13" 


Decoding words 


0.40 


0.34 


0.45 


6 


1,091 


75.82" 


Phonoiogicai STM 


0.39 


0.35 


0.43 


13 


1,911 


35.25" 


Arithmetic 


0.35 


0.30 


0.40 


8 


1,197 


40.64" 


Performance iQ 


0.34 


0.23 


0.45 


5 


253 


1.11 


Orai ianguage 


0.33 


0.30 


0.36 


30 


4,015 


323.33" 


Writing or writing name 


0.33 


0.26 


0.41 


4 


565 


1.11 


Visuai perception 


0.26 


0.21 


0.31 


9 


1,438 


64.54" 


Visuai motor 


0.22 


0.17 


0.27 


9 


1,333 


5.86 


Concept knowiedge 


0.20 


0.14 


0.26 


3 


873 


4.31 


Visuai memory 


0.17 


0.10 


0.23 


5 


875 


5.06 



* = p < 0.05: ** = p < 0.01. 



Variables reflecting early literacy skills that yielded average correlations that could be classified as 
moderate relationships appeared to fall into two groupings, one with larger correlations and one 
with smaller correlations. In the former category, measures of childrens AK yielded a moderate 
relationship of 0.48 averaged across 17 studies involving 2,038 children. Measures of print 
knowledge yielded a moderate relationship of 0.48 averaged across three studies involving 347 
children. Measures of childrens PA yielded a moderate relationship of 0.44 averaged across 20 
studies involving 2,461 children. Again, both types of rapid naming measures yielded moderate 
relationships. For measures involving rapid naming of letters or digits, an average correlation 
of 0.43 was obtained in three studies involving 333 children, and, for measures involving 
rapid naming of objects or colors, an average correlation of 0.42 was obtained in six studies 
involving 1,146 children. The second group of early literacy variables with lower correlations 
included phonological STM, oral language, and “writing or writing name.” In most cases, the 
average correlations of these variables were significantly lower than the other variables that 
yielded moderate to strong relationships with reading comprehension. Measures of children’s 
phonological STM yielded a correlation of 0.39 averaged across 13 studies involving 1,91 1 
children. Measures of oral language produced an average correlation of 0.33 in 30 studies 
involving 4,015 children. Measures of “writing or writing name” had an average correlation of 
0.33 across four studies involving 565 children. 
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Table 2.2a. Average Correlations for Prediction of Reading Comprehension by Variables Measured in 
Kindergarten or Earlier (predictor variables grouped conceptually) 



Predictor Variable 


Avg. r 


95% Cl 




N 

Studies 


N 

Children 


a 






Lower 

Bound 


Upper 

Bound 








AK 


0.48 


0.45 


0.51 


17 


2,038 


28.36* 


Concepts about print 


0.54 


0.48 


0.60 


3 


535 


3.31 


Print awareness 


0.48 


0.39 


0.56 


4 


347 


8.54* 


Decoding nonwords 


0.41 


0.30 


0.50 


3 


282 


72.13** 


Decoding words 


0.40 


0.34 


0.45 


6 


1,091 


75.82** 


Arithmetic 


0.35 


0.30 


0.40 


8 


1,197 


40.64** 


Performance IQ 


0.34 


0.23 


0.45 


5 


253 


1.11 


Concept knowledge 


0.20 


0.14 


0.26 


3 


873 


4.31 


Oral language 


0.33 


0.30 


0.36 


30 


4,015 


323.33** 


PA 


0.44 


0.41 


0.48 


20 


2,461 


58.08** 


Phonological STM 


0.39 


0.35 


0.43 


13 


1,911 


35.25** 


RAN letters and digits 


0.43 


0.34 


0.52 


3 


333 


1.46 


RAN objects and colors 


0.42 


0.38 


0.47 


6 


1,146 


6.78 


Readiness 


0.59 


0.51 


0.65 


3 


348 


11.76** 


Writing or writing name 


0.33 


0.26 


0.41 


4 


565 


1.11 


Visual perception 


0.26 


0.21 


0.31 


9 


1,438 


64.54** 


Visual motor 


0.22 


0.17 


0.27 


9 


1,333 


5.86 


Visual memory 


0.17 


0.10 


0.23 


5 


875 


5.06 



*=p<0.05: **=p<o.O'i. 



Additional variables that yielded at least moderate relationships with reading comprehension 
included measures of decoding nonwords (average r= 0.41 in three studies), measures of 
decoding words (average r = 0.40 in six studies), measures of arithmetic (average r = 0.35 in eight 
studies), and measures of performance IQ (average r = 0.34 in five studies). As with the results for 
decoding, correlations between measures of conventional literacy skills represent the continuity 
of reading skills. Measures of arithmetic knowledge and performance IQ are most likely an index 
of general cognitive abilities that are not specific to conventional literacy outcomes. All other 
variables yielded weak relationships with reading comprehension, including measures of general 
concept knowledge and all assessment measures involving visual skills. 

Similar to results for decoding outcomes, measures of global oral language skills were among 
the weakest predictors of reading comprehension. Examination of the CIs indicated that oral 
language was a significantly weaker predictor of reading comprehension than were measures of 
concepts about print, AK, PA, and the rapid naming tasks. Again, the Q statistics indicated that, 
for predictor variables with more than a handful of studies, there was significant heterogeneity in 
the observed sample of correlations. 

Although the panel searched widely for studies that had measured any early developing skills or 
abilities and then correlated these with any conventional literacy measures administered later on, 
most of the studies were conducted within a relatively brief window of development. Although 
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there were many studies that examined the relationship of skills measured at the beginning of 
kindergarten with those measured at the end of kindergarten, there were progressively fewer 
with outcomes measured when the children were older. This is not surprising, but it means that 
what is known about the relationship of these predictor variables with reading comprehension 
is limited to the very constrained conceptualizations of reading comprehension that can be 
measured with young children — levels of comprehension at which decoding is most likely to be 
implicated statistically. 



What Skills Measured in the Early Childhood Period or in Kindergarten Were Related 
to Spelling? 

The overall results for the predictive relations between variables measured in kindergarten or 
earlier and children’s spelling skills are presented in Tables 2.3 and 2.3a (Table 2.3 is organized 
by size of correlation, and Table 2.3a presents the same information organized conceptually by 
type of variable). Among the predictor variables that were most highly correlated with spelling 
outcomes were early measures of spelling, including invented spelling, and decoding. Two 
variables reflecting measures of spelling and two variables reflecting measures of decoding yielded 
strong relationships with children’s spelling skills measured in kindergarten and later. Measures 
of children’s AK also yielded a strong relationship with spelling outcomes of 0.54 averaged across 
18 studies involving 2,619 children. Additionally, measures of IQ and measures of arithmetic 
ability yielded strong relationships with spelling outcomes. As noted previously, measures of IQ 
and arithmetic knowledge most likely index general cognitive abilities that are not specific to 
conventional literacy outcomes. 

Moderate relationships with spelling outcomes were obtained for several variables. Variables 
reflecting children’s visual perceptual abilities yielded a correlation of 0.44 averaged across five 
studies involving 548 children. Measures of concepts about print had an average correlation 
of 0.43 in four studies involving 534 children. Children’s PA skills had an average correlation 
of 0.40 in 21 studies involving 2,522 children. Measures of global oral language skills yielded 
a moderate relationship of 0.36 averaged across 18 studies involving 2,087 children. Other 
variables that had average correlations classified as a moderate relationship with spelling included 
“writing or writing name” (average r= 0.36 in three studies), phonological STM (average r 
= 0.31 in 10 studies), and rapid naming of objects or colors (average r = 0.31 in six studies). 
Measures of performance IQ, visual motor skills, and environmental print yielded weak 
relationships with spelling outcomes. 

Examination of the CIs indicated that AK was the strongest literacy-specific nonconventional 
literacy predictor of spelling outcomes. With the exception of visual perceptual skills, all other 
variables that yielded a moderate relationship with spelling outcomes (e.g., concepts about print, 
PA, oral language) had significantly smaller average correlations with spelling than did AK. 
Among variables within the moderate relationship category, there were no significant differences 
in the strength of the correlations. The Q statistics again indicated that, for predictor variables 
with more than a handful of studies, there was significant heterogeneity in the observed sample 
of correlations. 
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Table 2.3. Average Correlations for Prediction of Spelling by Variables Measured in Kindergarten or Earlier 
(organized by size of correlation) 



Predictor Variable 


Avg. r 


95% Cl 




N 

Studies 


N 

Children 


a 






Lower 


Upper 












Bound 


Bound 








Spelling 


0.78 


0.74 


0.82 


4 


398 


85.99” 


Invented spelling 


0.69 


0.63 


0.74 


3 


354 


14.70” 


AK 


0.54 


0.51 


0.57 


18 


2,619 


62.73” 


Decoding nonwords 


0.54 


0.45 


0.63 


3 


246 


7.43* 


Decoding words 


0.54 


0.50 


0.58 


6 


1,112 


68.70” 


IQ 


0.54 


0.41 


0.65 


3 


142 


4.11 


Arithmetic 


0.50 


0.39 


0.60 


3 


203 


6.45* 


Visual perception 


0.44 


0.37 


0.51 


5 


548 


5.74 


Concepts about print 


0.43 


0.35 


0.49 


4 


534 


1 1 .32” 


PA 


0.40 


0.37 


0.44 


21 


2,522 


70.05” 


Oral language 


0.36 


0.32 


0.40 


18 


2,087 


13.73 


Writing or writing name 


0.36 


0.27 


0.44 


3 


397 


1.56 


Phonological STM 


0.31 


0.27 


0.36 


10 


1,520 


10.93 


RAN objects and colors 


0.31 


0.25 


0.36 


6 


1,132 


4.14 


Performance IQ 


0.29 


0.20 


0.37 


6 


446 


5.02 


Visual motor 


0.27 


0.18 


0.36 


3 


387 


3.09 


Environmental print 


0.25 


0.18 


0.31 


4 


818 


7.51 



* = p<0.05; ** = p<O.Ol 



Examination of Multivariate Studies 

These results indicate that a number of early literacy skills measured in kindergarten or earlier 
have strong to moderate relationships with conventional literacy outcomes. It is important to 
note, however, that all of the results presented in Tables 2.1, 2.2, and 2.3 represent zero-order 
correlations. It is possible that some of the observed relationships between early literacy variables 
and conventional literacy outcomes represent overlap among the early literacy variables. That 
is, variables may share predictive variance with each other; however, zero-order correlations do 
not allow this possibility to be investigated. Greater confidence that the observed zero-order 
correlation was attributable to the nominal construct of a predictor variable would be obtained if 
that variable were still significantly related to the outcome variable after other important variables 
were controlled. For example, does the correlation between PA and decoding still hold even when 
controlling for AK and oral language? 

To partially address this question, the panel examined the multivariate studies that were retrieved 
as part of the search for the first research question. Multivariate studies typically use multiple 
regression or similar analytic techniques to examine the predictive utility (i.e., semipartial 
correlation) of a variable in the context of other variables. Not all variables were examined in 
these multivariate studies, and, when they were, the control variables used encompassed a broad 
range of constructs (e.g., age, socioeconomic status, IQ, oral language, AK). In some cases, 
multiple variables were controlled at the same time in an analysis. In other cases, only a single 
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variable was controlled. There are no meta-analytical procedures for combining the results of 
multivariate analyses across studies; consequently, the results of the multivariate studies cannot 
be used to determine whether a variable would continue to be significantly related to one of 
the conventional literacy outcome variables under all potential conditions. However, multiple 
instances of robust prediction of a variable when different control variables are used either alone 
or in combination allow greater confidence in the results obtained from the analyses of the zero- 
order correlations. 



Table 2.3a. Average Correlations for Prediction of Spelling by Variables Measured in Kindergarten or 
Earlier (predictor variables grouped conceptually} 



Predictor Variable 


Avg. r 


95% Cl 




N 

Studies 


N 

Children 


a 






Lower 


Upper 












Bound 


Bound 








AK 


0.54 


0.51 


0.57 


18 


2,619 


Q2.7T 


Concepts about print 


0.43 


0.35 


0.49 


4 


534 


1 1 .32'* 


Environmental print 


0.25 


0.18 


0.31 


4 


818 


7.51 


Decoding nonwords 


0.54 


0.45 


0.63 


3 


246 


7.43* 


Decoding words 


0.54 


0.50 


0.58 


6 


1,112 


68.70" 


IQ 


0.54 


0.41 


0.65 


3 


142 


4.11 


Performance IQ 


0.29 


0.20 


0.37 


6 


446 


5.02 


Arithmetic 


0.50 


0.39 


0.60 


3 


203 


6.45* 


Qral language 


0.36 


0.32 


0.40 


18 


2,087 


13.73 


PA 


0.40 


0.37 


0.44 


21 


2,522 


70.05" 


Phonological STM 


0.31 


0.27 


0.36 


10 


1,520 


10.93 


RAN objects and colors 


0.31 


0.25 


0.36 


6 


1,132 


4.14 


Spelling 


0.78 


0.74 


0.82 


4 


398 


85.99" 


Invented spelling 


0.69 


0.63 


0.74 


3 


354 


14.70" 


Visual motor 


0.27 


0.18 


0.36 


3 


387 


3.09 


Visual perception 


0.44 


0.37 


0.51 


5 


548 


5.74 


Writing or writing name 


0.36 


0.27 


0.44 


3 


397 


1.56 



* = p <0.05: ** = p < 0.01. 



In general, the multivariate studies support the importance of the predictors identified as 
having moderate to strong relationships with conventional literacy outcomes in the meta- 
analyses. Multivariate studies examining the relationship between AK and decoding, reading 
comprehension, and spelling revealed that AK was a significant predictor when controlling for 
age, socioeconomic status, oral language, PA, and IQ. In multivariate studies, PA continued 
to be a significant predictor of decoding, reading comprehension, and spelling even when 
controlling for age, socioeconomic status, AK, oral language, IQ, and prior decoding ability. 
Measures of concepts about print were significant predictors of decoding and spelling when 
age and language were controlled but not when PA and AK were controlled in one study. 
Multivariate analyses of the predictive relations between phonological STM and decoding, 
reading comprehension, and spelling revealed that it continued to be a significant predictor when 
age, PA, oral language, socioeconomic status, and IQ were controlled. Oral language continued 
to be a significant predictor of decoding, reading comprehension, and spelling in some studies 
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when age, socioeconomic status, and IQ were controlled. In other studies, oral language was not 
a significant predictor of decoding when AK and PA were controlled. A few multivariate studies 
examined measures of “writing or writing name.” Even when controlling for AK, oral language, 
and IQ, measures of “writing or writing name” were significant predictors of decoding and 
comprehension. Multivariate studies showed that measures of rapid naming of either letters and 
digits or objects and colors were significant predictors of decoding, reading comprehension, and 
spelling after controlling for age, IQ, oral language, AK, PA, and phonological STM. Finally, the 
review of the multivariate studies did not support an independent predictive relationship 
for visual perceptual skills for spelling once socioeconomic status, IQ, AK, and oral language 
were controlled. 

Summary of Primary Analyses 

When measured in kindergarten or earlier, several variables are moderate to strong predictors 
of later outcomes in conventional literacy. A summary of the results of the three meta- analyses 
and a summary of findings from multivariate studies are shown in Table 2.4 for literacy-related 
variables with at least a moderate zero-order relationship with at least one conventional literacy 
outcome. Strength of relationship is based on the ratings discussed earlier (0-0.29 = small; 
0.30-0.49 = moderate; > 0.50 = strong). Ten variables meet this criterion. Of these 10 variables, 
six variables (AK, PA, rapid naming of letters and digits, rapid naming of objects and colors, 
“writing or writing name,” phonological STM) were consistently related to later conventional 
literacy outcomes, and these six variables continued to be predictive when other variables were 
controlled in multivariate analyses. Most of these findings are the result of a relatively large 
number of studies that included a large number of children. Consequently, these relationships 
between these variables and later conventional literacy outcomes not only are sizable, but they are 
likely to be highly reliable and stable. 



Table 2.4. Summary of Meta-Analytic and Multivariate Results for Literacy-Related Predictor Variables 
with Moderate to Strong Relationsbips with Conventional Literacy Outcomes 



Predictor Variable 


Decoding 


Reading 

Comprehension 


Spelling 


Multivariate 

Significance 


AK 


++ 


+ 


++ 


Yes 


PA 


+ 


+ 


+ 


Yes 


Concepts about print 


+ 


++ 


+ 


Sometimes 


RAN letters and digits 


+ 


+ 


NA 


Yes 


RAN objects and colors 


+ 


+ 


+ 


Yes 


Writing or writing name 


+ 


+ 


+ 


Yes 


Oral language 


+ 


+ 


+ 


Sometimes 


Phonological STM 


— 


+ 


+ 


Yes 


Visual perception 


— 


— 


+ 


No 


Print awareness 


— 


+ 


NA 


NA 



Note: ++ = strong relationship based on zero-order correlations: + = moderate relationship based on zero-order correlations: — = vveak relationship 
based on zero-order correlations: NA = no relevant data available for analysis. 
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These results also identify variables that have only a weak relationship with later conventional 
literacy or for which there is no current evidence of a relationship. In general, variables reflecting 
measures of visual skills (i.e., visual motor, visual memory) are only weakly related to later 
reading and writing. Additionally, variables reflecting measures of environmental print (e.g., the 
ability to decode or read common signs and logos) are only weakly related to later reading and 
writing. Some variables that have been proposed as reflecting precursor literacy skills did not 
appear in any of the analyses (e.g., emergent reading). As noted previously, only those variables 
for which at least three studies included a reported correlation between the skill measured in 
kindergarten or earlier and a conventional literacy outcome could be included in the analyses. It 
may be that these variables have not been related to later conventional literacy outcomes or that 
there were fewer than three studies available in the published literature. 

A final point to take away from these results concerns the remarkable degree of stability of 
conventional literacy skills. Generally, these skills were the strongest or among the strongest 
predictors of decoding, reading comprehension, and spelling. Moreover, the sizes of the 
relationships were in the strong range, accounting for 27 to 6 1 percent of the variance in later 
measures of decoding and spelling. These findings highlight the fact that the origins of well- 
developed conventional literacy skills are found very early in children’s educational experience, 
and these findings are consistent with studies (Francis et ah, 1996; Juel, 1988; Torgesen & 
Burgess, 1998) showing that the consequences of falling seriously behind in the development of 
conventional literacy skills are likely to be long-lasting in the absence of substantial remedial efforts. 



Secondary Analyses 

The results reported in the primary analyses address the primary question that the panel posed. 
That is, they identified the skills and abilities measured when children are in kindergarten 
or earlier that have moderate to strong predictive relations to later outcomes in conventional 
literacy skills. However, these findings represented a broad age range of when the predictor skills 
were assessed (i.e., birth through kindergarten) and when the conventional literacy outcome 
was assessed (i.e., kindergarten and later) and variable lengths of time between predictors and 
outcome assessments. Additionally, several of the predictor variables were grouped into very 
broad categories. In some cases, there are compelling theoretical reasons for stronger relations 
between some components of these skills and conventional literacy outcomes than others. 
Therefore, secondary analyses were conducted to address questions related to age of assessment 
for both the predictor variable (i.e., preschool versus kindergarten) and the outcome variable 
(i.e., kindergarten versus first or second grade). Secondary analyses also afforded a finer-grained 
analysis of some of the predictor variables (i.e., oral language, PA). 

Do Variables Have Stronger or Weaker Predictive Relations Depending on When They 
Were Measured (Preschool Versus Kindergarten)? 

As noted in the preceding section, the relationships between the predictor variables and 
conventional literacy outcomes were estimated using a potentially broad age range of children. 
In fact, very few studies included in the analyses investigated very young children. Most of 



68 



DEVELOPING EARLY LITERACY: REPORT OF THE NATIONAL EARLY LITERACY PANEL 



the children in the samples of the studies used in the analyses were four years of age or older. 
Nonetheless, developmental or experiential factors might differentially influence the size of the 
predictive relationships for variables measured at the preschool age and those for children already 
in kindergarten. For instance, many children who are in kindergarten are already exposed to 
formal reading instruction. It is possible that this instructional exposure both develops one of the 
predictor skills and is responsible for the predictive relationship. For instance, a child exposed to 
high-quality reading instruction may develop strong PA and strong decoding skills; however, in 
the absence of this instruction, neither PA nor decoding would be related. 

To examine the impact of the age at which a skill was assessed on the predictive relationship 
with later conventional literacy skills, studies were divided into two groups. One group included 
studies in which all of the children were not in kindergarten (i.e., children were preschool age) 
and one group included studies in which all of the children were in kindergarten. Studies with 
blended samples were excluded from the analyses (i.e., studies that included both preschool- 
and kindergarten-age children in the same analyses were not used in these analyses). Again, 
these meta-analyses require that a minimum of three studies be available to estimate an ES. 
Consequently, any predictor-outcome combination that was not represented by at least three 
preschool studies and at least three kindergarten studies could not be included in these analyses. 

Results of the analyses for the decoding outcome are shown in Table 2.5. As can be seen in the 
table, there were few differences in the predictive relationship between a variable and decoding 
dependent on the age of assessment. In most cases, there was substantial overlap of the CIs for 
the separate ES estimates, indicating that the predictive relationship was similar regardless of 
when the predictor variable was measured. Three variables did differ significantly or marginally 
depending on when they were assessed. Both phonological STM and visual perception skills 
had significantly stronger relationships with decoding outcomes when they were measured in 
preschool than when they were measured in kindergarten. In contrast, the predictive relationship 
between “writing or writing name” and decoding was marginally higher when measured in 
kindergarten than when measured in preschool. 

Results of the analyses for the reading comprehension outcome are shown in Table 2.6. Both AK 
and oral language were equally predictive of reading comprehension regardless of measurement 
in preschool versus kindergarten. PA was a marginally better predictor of reading comprehension 
when it was measured in kindergarten than when it was measured in preschool (although both 
correlations reflect moderate relationships). In contrast, both phonological STM and visual 
perception skills were significantly better predictors of reading comprehension when they were 
measured in preschool than when they were measured in kindergarten. Eor phonological STM, 
the preschool correlation represented a strong relationship. 

Results of the analyses for the spelling outcome are shown in Table 2.7. Only the predictive 
relation for AK significantly varied as a function of preschool or kindergarten measurement 
points. When measured in kindergarten, AK had a strong relationship with spelling, but when 
measured in preschool, it had a moderate relationship with spelling. 
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Table 2.5. Average Correlations for Prediction of Decoding by Variables Measured in Prescbool versus 
Kindergarten 





Studies Including Only Prescbool Children 




Studies Including Only Kindergarten Children 


Predictor 

Variable 


Avg. r 


95% Cl 


N 

Studies 


N 

Children 




Avg. r 


95% Cl 


N 

Studies 


N 

Children 


Comparison 






Lower 

Bound 


Upper 

Bound 










Lower 

Bound 


Upper 

Bound 








AK 


0.49 


0.44 


0.53 


7 


990 


0.50 


0.48 


0.52 


46 


6,865 


P = K 


IQ 


0.44 


0.33 


0.54 


3 


233 


0.45 


0.41 


0.48 


9 


1,768 


P = K 


PA 


0.43 


0.37 


0.49 


11 


820 


0.40 


0.38 


0.42 


60 


7,805 


P = K 


Phonological 

STM 


0.42 


0.35 


0.48 


8 


663 


0.24 


0.21 


0.27 


26 


4,270 


P>K 


Writing or 
writing name 


0.41 


0.33 


0.48 


3 


509 


0.52 


0.48 


0.57 


7 


1,141 


K>P 


Visual perception 


0.39 


0.30 


0.47 


3 


384 


0.19 


0.15 


0.23 


13 


2,167 


P>K 


Concepts about 
print 


0.34 


0.25 


0.43 


4 


393 


0.34 


0.30 


0.38 


8 


2,211 


P = K 


Oral language 


0.32 


0.28 


0.37 


15 


1,377 


0.33 


0.31 


0.35 


50 


8,131 


P = K 


Performance IQ 


0.32 


0.19 


0.43 


5 


228 


0.30 


0.26 


0.33 


11 


2,635 


P = K 


Visual motor 


0.28 


0.19 


0.36 


4 


435 


0.24 


0.18 


0.30 


10 


881 


P = K 




Note: P= preschool correlation: K 


= kindergarten correlation. 
















Table 2.6. Average Correlations for Prediction of Reading Comprebension by Variables Measured in 
Prescbool Versus Kindergarten 




Studies Including Only Prescbool Children 


( 


Studies Including Only Kindergarten Children 




Predictor 

Variable 


Avg. f 


95% Cl 


N 

Studies 


N 

Children 




Avg. f 


95% Cl 


N 

Studies 


N 

Children 


Comparison 






Lower 

Bound 


Upper 

Bound 










Lower 

Bound 


Upper 

Bound 








Phonological 

STM 


0.51 


0.44 


0.57 


4 


479 


0.34 


0.30 


0.39 


9 


1,433 


P>K 


AK 


0.45 


0.38 


0.51 


4 


583 


0.48 


0.44 


0.52 


14 


1,638 


P = K 


Visual 

perception 


0.41 


0.32 


0.49 


3 


378 


0.21 


0.15 


0.27 


6 


1,060 


P>K 


Oral language 


0.40 


0.33 


0.47 


7 


575 


0.32 


0.29 


0.35 


23 


3,441 


P = K 


PA 


0.36 


0.27 


0.43 


5 


464 


0.46 


0.43 


0.50 


15 


1,998 


K>P 



Note: P= preschool correlation: K = kindergarten correlation. 
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Table 2.7. Average Correlations for Prediction of Spelling by Variables Measured in Prescbool Versus 
Kindergarten 



Studies Including Only Preschool Children 




Studies Including Only Kindergarten Children 


Predictor , N N 

Variahle ^ ° Studies Children 




A,a.r 95% Cl Comparison 


Lower Upper 
Bound Bound 




Lower Upper 
Bound Bound 



AK 


0.43 


0.37 


0.49 


3 


668 


0.55 


0.52 


0.58 


16 


2,288 


K>P 


Oral language 


0.36 


0.29 


0.43 


6 


621 


0.36 


0.31 


0.40 


12 


1,466 


P = K 


PA 


0.38 


0.31 


0.45 


6 


612 


0.42 


0.38 


0.46 


17 


2,092 


P = K 


Phonological 

STM 


0.40 


0.32 


0.47 


4 


472 


0.27 


0.22 


0.33 


6 


1,048 


P = K 



Note: P = preschool correlation: K = kindergarten correlation. 



In sum, there appears to be little difference in the size of the predictive relationship for variables 
measured in preschool and those measured in kindergarten. Of the comparisons that could 
be examined, 75 percent yielded results that did not differ significantly between the preschool 
assessment and the kindergarten assessment. Of the four significant differences, only the 
AK-spelling effect represented one of the consistently strong predictor variables (see Table 2.4). 
Moreover, when differences were significant, they tended to favor the preschool assessment, and it 
was typically the case that correlations for both preschool and kindergarten assessments remained 
in the moderate relationship range. 

Do Variables Have Stronger or Weaker Predictive Relations Depending on When the 
Conventional Literacy Dutcome Variable Was Measured (Kindergarten Versus First or 
Second Grade)? 

Similar to the question of differential effects as a function of the age at which the predictor 
variable was assessed, it is possible that the age at which the conventional literacy outcome 
was assessed affected the strength of the predictive relationship between a skill measured in 
kindergarten or earlier and an outcome in the conventional literacy domain. For instance, it is 
likely that assessments that occur closer in time will be more highly correlated than assessments 
with more time elapsed between them. The primary analyses included outcomes measured from 
kindergarten through later grades. Although most studies included outcomes measured early in 
the elementary-school period, some studies included much longer-term outcomes. 

To examine the impact of the age at which a conventional literacy outcome was assessed on the 
predictive relationship with skills measured when children were in kindergarten or younger, 
studies were divided into two groups. One group included studies in which all of the children 
were in kindergarten at the time of the outcome assessment, and one group included studies in 
which all of the children were in first or second grade at the time of the outcome assessment. 
Studies with blended samples were excluded from the analyses (i.e., studies that included 
outcomes measured and reported for a combined kindergarten-first- or second-grade sample 
were not used in these analyses). Again, these meta- analyses require that a minimum of three 
studies be available to estimate an ES. Consequently, any predictor-outcome combination that 
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was not represented by at least three kindergarten studies and at least a total of three first-grade or 
second-grade studies could not be included in these analyses. 

Comparisons of the predictive relationships between skills measured in kindergarten or earlier 
and decoding measured in either kindergarten or first or second grade are shown in Table 2.8. 
There were a number of significant differences that generally showed that the kindergarten 
correlation was higher than the first-grade-second-grade correlation (i.e., AK, arithmetic, 
concepts about print, decoding words, decoding nonwords, visual perception skills). Two 
correlations (phonological STM and rapid naming of letters or digits) were higher when 
decoding was measured in first or second grade than when it was measured in kindergarten. 

With the exceptions of the correlation between visual perception skills and decoding measured in 
first or second grade and the correlation between phonological STM and decoding measured in 
kindergarten, all correlations were in the strong to moderate relationship range regardless of the 
age of outcome assessment. 

For reading comprehension, only a sufficient number of studies measured the outcome in 
kindergarten for the oral language variable. This comparison indicated that oral language was a 
significantly stronger predictor when reading comprehension was measured in first or second grade 
than when it was measured in kindergarten, most likely because of the difficulty of obtaining 
a valid assessment of reading comprehension in kindergarten. Results for spelling measured in 
kindergarten versus first or second grade are shown in Table 2.9. Only the correlation between AK 
and spelling varied as a function of when spelling was assessed. AK had a strong relationship with 
spelling measured in kindergarten, whereas it had a moderate relationship with spelling measured 
in first or second grade. 

In contrast to results comparing age of assessment for the predictor variable, age of assessment of 
the conventional literacy outcome variable had a significant influence on the size of the predictive 
relationship. In these comparisons, 50 percent of the correlations differed significantly depending 
on when the outcome variable was measured. In all but two cases, the correlation was stronger 
when the outcome variable was assessed in kindergarten. The simplest explanation for this 
finding is proximity of assessment. That is, two assessments occurring closer in time are likely to 
be more highly correlated than two assessments separated by years. Between kindergarten and 
first and second grades, children are exposed to a significant amount of instruction in reading, 
which varies by school. The greater the amount of time between an initial assessment and 
the measurement of the outcome, the more that instructional intensity and quality can affect 
children’s scores. Regardless, as was the case for the assessment of the predictor variable, the size 
of the correlations when conventional literacy skills were measured either in kindergarten or in 
first or second grade reflected moderate to strong relationships. 

Do Variations in the Aspect of Oral Language Measured Make a Difference in the 
Strength of the Predictive Relationship (e.g., vocabulary versus grammar)? 

Results from the primary analyses revealed a moderate relationship between oral language skills 
and conventional literacy outcomes. Although the average correlation was in the moderate 
range, it was among the weakest predictors in that range (average r = 0.33 to 0.36). Additionally, 
the predictive relationship of oral language did not consistently hold up when other predictor 
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variables were controlled in multivariate analyses. Whereas this finding was not entirely 
surprising for decoding and spelling, it was somewhat unexpected that the predictive relationship 
between oral language and reading comprehension was at the low end of the moderate range 
(see, e.g., Senechal & LeFevre, 2002; Storch & Whitehurst, 2002; Whitehurst & Lonigan, 

1998). In the primary analyses, oral language was a global variable representing measures of 
different components of oral language skill (e.g., vocabulary, listening comprehension, syntax). 

It is possible that different aspects of oral language have weaker or stronger relationships with 
different aspects of conventional literacy. 



Table 2.8. Average Correlations for Prediction of Decoding Measured in Kindergarten Versus First or 
Second Grade 





Decoding Measured in Kindergarten 


Decoding Measured in First or Second Grade 




Predictor 

Variable 


Avg. f 


95% Cl 


N 

Studies 


N 

Children 


Avg. f 


95% Cl 


N 

Studies 


N 

Children 


Comparison 






Lower 

Bound 


Upper 

Bound 








Lower 

Bound 


Upper 

Bound 








AK 


0.58 


0.56 


0.61 


24 


3,142 


0.44 


0.42 


0.47 


30 


4,374 


K>Gl/2 


Arithmetic 


0.54 


0.51 


0.57 


6 


2,328 


0.39 


0.36 


0.42 


8 


2,824 


K>Gl/2 


Concepts about 
print 


0.53 


0.43 


0.62 


3 


230 


0.33 


0.29 


0.36 


8 


2,129 


K>G1/2 


Decoding 

nonwords 


0.80 


0.77 


0.83 


5 


481 


0.56 


0.47 


0.64 


3 


282 


K>Gl/2 


Decoding words 


0.69 


0.66 


0.71 


10 


1,620 


0.50 


0.46 


0.53 


10 


1,945 


K>G1/2 


Invented spelling 


0.49 


0.41 


0.57 


5 


351 


0.63 


0.57 


0.69 


5 


427 


K = G1/2 


Oral language 


0.32 


0.29 


0.35 


26 


3,985 


0.33 


0.30 


0.35 


41 


5,534 


K = G1/2 


Performance IQ 


0.32 


0.27 


0.36 


6 


1,365 


0.28 


0.23 


0.32 


10 


1,603 


K = G1/2 


PA 


0.42 


0.40 


0.45 


34 


3,618 


0.42 


0.40 


0.44 


38 


5,206 


K = G1/2 


Phonological STM 


0.22 


0.18 


0.27 


12 


1,633 


0.31 


0.28 


0.34 


21 


3,278 


K<Gl/2 


RAN letters 
and digits 


0.36 


0.31 


0.41 


6 


1,251 


0.52 


0.46 


0.56 


7 


892 


K<G1/2 


RAN objects 
and colors 


0.28 


0.20 


0.35 


5 


578 


0.33 


0.29 


0.36 


10 


2,305 


K = G1/2 


Visual perception 


0.34 


0.25 


0.43 


4 


421 


0.14 


0.09 


0.19 


9 


1,807 


K>G1/2 



Note: K = correlation with kindergarten outcome; G i/2 = correiation with first- or second-grade outcome. 
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Table 2.9. Average Correlations for Prediction of Spelling Measured in Kindergarten Versus First or Second 
Grade 



Spelling Measured in Kindergarten 




Spelling Measured in First or Second Grade 


Predictor , N N 

Variable ° Studies Children 




A.S., 95% Cl Comparison 


Lower Upper 
Bound Bound 




Lower Upper 
Bound Bound 



AK 


0.60 


0.57 


0.64 


10 


1,361 


0.49 


0.45 


0.53 


10 


1,503 


K>G1/2 


Oral language 


0.37 


0.31 


0.42 


7 


1,142 


0.35 


0.30 


0.40 


12 


1,073 


K = G1/2 


PA 


0.43 


0.38 


0.47 


10 


1,305 


0.41 


0.37 


0.46 


11 


1,351 


K = Gl/2 


Phonological 

STM 


0.28 


0.22 


0.34 


4 


902 


0.37 


0.31 


0.44 


6 


714 


K = Gl/2 


RAN objects 
and colors 


0.29 


0.22 


0.36 


3 


738 


0.34 


0.25 


0.42 


3 


394 


K = Gl/2 



Note: K = correlation with kindergarten outcome: G 1/2 = correlation with first- or second-grade outcome. 



To assess the possibility that different aspects of oral language are more or less related to outcomes 
in conventional literacy, all of the oral language measures were coded into subcategories reflecting 
the content of the assessment measure used. How these subcategories related to decoding and 
reading comprehension was examined. Spelling was not included in these analyses because of 
the limited number of studies available (i.e., 18 studies for spelling versus 63 for decoding and 
30 for reading comprehension). The results of these analyses are shown in Table 2.10. As is clear 
from the table, some aspects of oral language had substantial correlations with decoding and 
reading comprehension. Composite measures that assess multiple aspects of oral language, such as 
vocabulary, syntax, and listening comprehension, in the same test had correlations in the strong 
relationship range for both decoding and reading comprehension. The overall language composite 
measures (e.g., such tests as the Preschool Language Scale [Zimmerman, Steiner, & Pond, 2002] 
or the Clinical Evaluation of Language Fundamentals: Preschool [Wiig, Semel, & Secord, 1992]) 
accounted for almost 50 percent of the variance in reading comprehension, on average. However, 
it is not clear whether the strength of this relationship reflects the constructs assessed by the 
measure or the increased reliability of a composite measure of substantial length. 

Notably, measures of vocabulary had relatively weak relationships with both decoding and 
reading comprehension, falling either into the low end of the moderate range or into the 
weak range. Measures of more complex oral language skills, such as grammar, definitional 
vocabulary, and listening comprehension, had stronger relationships with both decoding and 
reading comprehension, falling either into the mid to high moderate range or into the strong 
range. For reading comprehension, measures of grammar, definitional vocabulary, and listening 
comprehension were generally significantly stronger predictors than were measures of vocabulary. 
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Table 2.10. A verage Correlations for Prediction of Decoding and Reading Comprehension by Subcategories 
of Oral Language Measures 



Oral Language Measure 




Decoding 


Reading Comprehension 


Comparison 


Avg. f 


95% Cl 


Avg. f 


95% Cl 




Lower 

Bound 


Upper 

Bound 




Lower 

Bound 


Upper 

Bound 


Overall language comprehension 


0.58 


0.52 


0.63 


0.70 


0.66 


0.74 


D<RC 


Receptive language comprehension 


0.52 


0.46 


0.58 


0.63 


0.58 


0.68 


D = RC 


Expressive language comprehension 


0.48 


0.43 


0.53 


0.59 


0.54 


0.64 


D<RC 


Grammar 


0.47 


0.43 


0.50 


0.64 


0.59 


0.68 


D<RC 


Definitional vocabulary 


0.38 


0.30 


0.46 


0.45 


0.36 


0.53 


D = RC 


Verbal knowledge 


0.36 


0.31 


0.41 


0.45 


0.39 


0.51 


D = RC 


Verbal IQ 


0.35 


0.31 


0.39 


0.35 


0.25 


0.45 


D = RC 


Receptive vocabulary 


0.34 


0.31 


0.37 


0.25 


0.19 


0.31 


D = RC 


Listening comprehension 


0.33 


0.30 


0.36 


0.43 


0.38 


0.48 


D<RC 


Vocabulary NOS 


0.33 


0.28 


0.38 


0.31 


0.26 


0.36 


D = RC 


Expressive vocabulary 


0.24 


0.16 


0.31 


0.34 


0.27 


0.40 


D = RC 


Language NOS 


0.20 


0.13 


0.26 


0.31 


0.26 


0.37 


D = RC 



Note: D = decoding; RC = reading comprehension. 



Such results are potentially instructive about the focus of early childhood education. They suggest 
that a focus on building vocabulary alone is unlikely to be sufficient for improving outcomes 
not only in literacy but also in oral language itself Although, these results should not be taken 
to imply that well-developed vocabularies are unimportant ior literacy. The results suggest that 
well-developed vocabularies are insujficient for literacy. More complex oral language skills are 
dependent on vocabulary. For instance, a child with strong grammatical knowledge but limited 
vocabulary would have a difficult time understanding a text or writing a meaningful narrative. 
Vocabulary provides the foundation for grammatical knowledge, definitional vocabulary, and 
listening comprehension. 

Do Variations in the Aspect of Phonological Awareness Measured Make a Difference 
in the Strength of the Predictive Relationship (e.g., level of linguistic complexity, type of 
task used)? 

Perhaps no area in early literacy has generated as much theoretical debate and empirical inquiry as 
has the topic of PA. Developing PA is often seen as a necessary condition for acquiring decoding 
skills in alphabetic languages. Deficits in PA in older children are considered one of the primary 
causes of developmental dyslexia (Stanovich, 1988; Stanovich & Siegel, 1994). Much of the debate 
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in this area has focused on the level of PA required to develop decoding skills and the ways in which 
PA can be demonstrated. For instance, some have argued that only phonemic awareness is related 
to reading and that only measures that involve manipulation of speech sounds reflect this skill 
because these measures require reflection on abstract representations. Generally, tasks that require 
manipulation of phonemes are too difficult for young children, and tasks requiring manipulation 
rather than detection of speech sounds are often also too difficult for young children. Consequently, 
by this definition of PA, young children cannot demonstrate reading-related PA. 

The primary analyses addressed the relationship between measures of the global construct of PA, 
measured in different ways at different levels of abstraction. PA is often seen as developing on a 
continuum, starting with sensitivity to large and concrete units of sounds (i.e., words, syllables) 
and progressing to sensitivity to small and abstract units of sounds (i.e., phonemes; see Lonigan, 
2006). This developmental progress is usually described as occurring along the dimension of 
linguistic complexity. Additionally, many different tasks have been used to assess PA, ranging 
from simple detection tasks (e.g., identification of rhyming words) to tasks requiring synthesis 
of sounds (e.g., blending phonemes together to make a word) to tasks requiring analysis of 
components of words (e.g., segmenting words, counting phonemes). 

To address the possibility that different aspects of PA, in terms of level of either linguistic 
complexity or cognitive operation required, have stronger or weaker relations with conventional 
literacy outcomes, all of the PA measures were coded into subcategories. One subcategory 
coding reflected the level of linguistic complexity (i.e., phoneme, subphoneme, rhyme, or a 
combination). A second subcategory coding reflected the type of cognitive operation required 
(i.e., identification, synthesis, analysis, or a combination). Again, analyses were restricted to 
outcomes involving either decoding or reading comprehension because of the limited number of 
studies that included spelling as an outcome. 

Results for the different levels of linguistic complexity assessed by PA tasks are shown in Table 

2. 1 1 . As noted in the table, the results were not influenced by whether the outcome measure 
was decoding or reading comprehension. As with the analyses of the oral language subcategories, 
the composite PA measures had the highest predictive relationship with decoding. In terms of 
the specific levels of linguistic complexity, phonemic awareness had the highest correlation with 
decoding and reading comprehension; however, the difference between phonemic awareness 
and subphonemic awareness (e.g., syllable awareness) was not significant. For decoding but not 
reading comprehension, phoneme awareness was a significantly stronger predictor than was 
rhyme awareness. Correlations for rhyme awareness and subphoneme awareness did not differ 
significantly from each other for either decoding or comprehension. 

Results for the different cognitive operations required on the PA measures are shown in Table 

2.12. With the exception of the strength of the correlation for identity tasks, the results were 
similar for decoding and reading comprehension. Again, the composite measure was the strongest 
predictor of decoding. For decoding, analysis tasks were a significantly stronger predictor 

than either synthesis tasks or identity tasks. For reading comprehension, analysis tasks were a 
significantly stronger predictor than synthesis tasks but did no better than identity tasks. The 
correlations for synthesis tasks were not significantly different from the correlations for identity 
tasks for either decoding or reading comprehension. 
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Table 2.11. A verage Correlations for Prediction of Decoding and Reading Comprehension by Variables 
Measuring Phonological Awareness at Different Levels of Linguistic Complexity 



Level of Linguistic Complexity 




Decoding 




Reading Comprehension 




Avg. r 


95% Cl 


Avg r 


95% Cl 


Comparison 






Lower 

Bound 


Upper 

Bound 




Lower 

Bound 


Upper 

Bound 




Overall 


0.39 


0.38 


0.40 


0.42 


0.39 


0.45 


D = RC 


Composite 


0.47 


0.43 


0.50 


0.36 


0.29 


0.44 


D = RC 


Phoneme 


0.42 


0.39 


0.44 


0.44 


0.38 


0.48 


D = RC 


Subphoneme 


0.36 


0.33 


0.39 


0.44 


0.39 


0.48 


D = RC 


Rhyme 


0.29 


0.25 


0.33 


0.38 


0.29 


0.47 


D = RC 



Note: D = decoding; RC = reading comprehension. 



Table 2. 12. Average Correlations for Prediction of Decoding and Reading Comprehension by Variables 
Measuring Phonological Awareness Using Different Cognitive Dperations 



Cognitive Operation 




Decoding 


Reading Comprehension 


Comparison 


Avg. f 


95% Cl 


Avg f 


95% Cl 




Lower 

Bound 


Upper 

Bound 




Lower 

Bound 


Upper 

Bound 


Overall 


0.39 


0.38 


0.41 


0.42 


0.39 


0.44 


D = RC 


Composite 


0.48 


0.45 


0.52 


0.44 


0.33 


0.54 


D = RC 


Analysis 


0.44 


0.42 


0.46 


0.47 


0.42 


0.51 


D = RC 


Synthesis 


0.36 


0.32 


0.39 


0.31 


0.24 


0.38 


D = RC 


Identity 


0.31 


0.28 


0.34 


0.40 


0.36 


0.45 


D<RC 



Note: D = decoding; RC = reading comprehension 



As with the findings from the analysis of the subcategories of oral language skills, these results are 
potentially instructive for early childhood education. In contrast to the results from the analysis 
of the subcategories of oral language skills, however, these findings do not suggest a strong 
division between the types of PA demonstrated by children with respect to later conventional 
literacy skills. That is, the specific level of linguistic complexity mattered little in the strength 
of the relation to conventional literacy skills, although measures of rhyme were consistently the 
weakest predictor. What is likely more important is that assessment and instructional activities 
occur within a child’s developmental level along the developmental continuum of PA. There was 
a consistent and significant advantage for analysis tasks over synthesis tasks and identity tasks. 
This likely indicates a deeper ability to manipulate the sounds in words, reflecting a level closer 
to achieving the alphabetic principle. 



Overall Summary 

These results provide compelling evidence as to what some of the important early developing 
precursor skills are to reading, writing, and spelling development. Across three different outcome 
domains — decoding, reading comprehension, and spelling — a consistent collection of predictor 
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variables emerged that possess moderate to strong relationships to these important outcomes. 

In many cases, these variables provided significant prediction of later literacy outcomes even 
when other variables were controlled. Based on these findings, there is strong evidence for the 
importance of AK, PA, rapid naming tasks, “writing or writing name,” and phonological STM 
as predictors of later reading and writing skills. Less consistent evidence exists for the importance 
of oral language and concepts about print as predictors of later reading and writing skills, 
mainly because these variables do not always continue to predict literacy outcomes once other 
variables, such as AK or PA, are controlled. There was weak evidence for the importance of visual 
perception skills as a predictor of later reading and writing skills, because a moderate relationship 
emerged for only one outcome variable and because it did not continue to predict literacy 
outcomes once other variables, such as AK or PA, are controlled. 

The secondary analyses revealed that the important predictor variables continued to have 
moderate to strong relationships with later measures of literacy regardless of the age at which 
the predictor variable was assessed (e.g., preschool versus kindergarten) or the age at which the 
outcome variable was assessed (e.g., kindergarten versus first or second grade). Although there 
were some minor differences involving age of assessment of the predictor variable, age did not 
influence the strongest predictor variables. Greater differences were observed depending on 
when the outcome assessments were administered; generally, there were higher correlations with 
kindergarten outcomes than with first- or second-grade outcomes. However, this is most likely 
due to the closer time proximity of these assessments than to age differences, per se. 

Implications for Research and Practice 

These findings have implications for practices in early childhood education. First, the pattern of 
findings is strikingly consistent and identifies key skills that can serve as important, reliable, and 
stable indicators for identifying childrens development toward acquiring conventional literacy 
skills. Early childhood educators interested in monitoring childrens progress or in identifying those 
children who need targeted intervention to promote early literacy skills should use assessments that 
provide reliable and valid measurement of these skills. The findings also suggest that instruction 
focused on these skills may provide valuable literacy preparation, particularly for children at risk for 
developing reading difficulties. These findings provide guidance to early childhood educators for 
selecting appropriate curricula for the children they serve, and they provide guidance to curriculum 
developers concerning the skills that should be targeted within instructional activities. 

The results suggest a need for more careful study of the role of oral language in literacy 
development. Some aspects of oral language were clearly more strongly related to later literacy 
outcomes than were other aspects of oral language. Notably, measures of simple vocabulary 
knowledge were fairly weak predictors of later decoding and reading comprehension, and these 
measures tended to not remain significant when other variables were included in multivariate 
analyses. In contrast, more complex aspects of oral language, such as grammar, definitional 
vocabulary, and listening comprehension, had more substantial predictive relations with later 
conventional literacy skills. These results suggest that an instructional focus on vocabulary during 
the preschool and kindergarten years is likely a necessary but insufficient approach to promoting 
later literacy success. 
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The value of these variables for predicting later literacy success is without question, and future 
research could help to provide systematic investigation into which combinations of predictors 
would work best in various contexts. There is less certainty that teaching these variables early on 
will result in later achievement improvement. This is because these studies provide correlational 
data, and such data are not sufficient for determining a causal connection between these factors 
and later learning. 

Results from the analysis of findings related to PA appear to have instructional implications 
for early childhood educators. These findings suggest the importance of attending to children’s 
progress along a developmental continuum of PA, rather than an emphasis on particular PA 
skills. These analyses did not reveal important differences in phonological memory, synthesis, or 
segmentation. However, they do suggest an order to the development of all of these skills across 
a progression of smaller and smaller units of sound. Rather than trying to teach any particular 
skill (such as phonological STM), it may be of greater value to ensure that progress is occurring 
and that children are becoming progressively more able to deal with smaller and smaller units of 
sound (e.g. words, syllables, onset rimes, phonemes). 

In addition to more research into the role of oral language in literacy development, additional 
work concerning the causal status of these predictor variables is needed. One avenue of research 
might examine the predictive significance of the variables identified as consistent predictors of 
conventional literacy outcomes in the context of each other. Future research could profitably 
examine interventions designed to promote development of the skills identified here; such 
investigations should both explore the impact of the interventions on the development of the 
proximal skills and, most importantly, examine how children fare on distal conventional literacy 
outcomes. Although the upcoming chapters of this report will show what is known about the 
effectiveness of various approaches for teaching some of these skills (as well as conventional 
literacy skills), this work is limited to studies already in the published literature, most of which 
do not report longer-range follow-ups. 
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The National Early Literacy Panel (NELP) identified 83 studies that examined the effectiveness 
of various interventions that attempted to teach children code-related skills. Interventions in 
this category focused on teaching aspects of the alphabetic principle (i.e., the knowledge that 
letters in written words represent the sounds in spoken words). This was the largest collection 
of intervention studies that the panel reviewed, and it included interventions aimed at the 
development of phonological awareness (PA), alphabet knowledge (AK), and early decoding 
skills (i.e., phonics). The studies analyzed in this chapter are listed at the end of the chapter. 

Most often, the children in these studies were attending some type of preschool or kindergarten 
program, and the interventions were implemented in addition to whatever education activities 
were already part of their preschool or kindergarten experiences. Children in the comparison 
groups usually received the regular (often unspecified) activities of their preschool or kindergarten 
programs, but sometimes the comparison-group children received an alternative intervention 
designed to provide a specific contrast to the code-focused teaching. 

As described in the preceding chapter, the criterion for inclusion of a study into the NELP 
analyses of intervention effects included (a) use of a group-comparison design (randomized 
control trial [RCT] or quasiexperimental design [QED]), (b) use of outcome measures that 
were assessments either of a conventional literacy skill (i.e., decoding, reading comprehension, 
spelling) or of one of the skills that the NELP analyses identified as a predictor of later literacy 
skills, and (c) reported sufficient information to allow an effect size (ES) to be calculated. Studies 
that used a QED were required to have evidence of initial group comparability (i.e., the groups 
had pretest scores within 0.5 standard deviation of each other). 
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Virtually all studies in this category of interventions included some form of PA training. These 
interventions involved training children either individually or in small groups to identify sounds in 
words (e.g., match words with the same initial sound) or, more often, to manipulate sounds in words 
(e.g., combine sounds to form words, segment or delete parts of words). In some studies, these PA 
training activities were combined with other code-focused training activities, forming two broad 
categories of combined interventions. One category of combined interventions included studies 
in which the activities included both PA training and training activities designed to teach children 
AK, such as letter names or, occasionally, both letter names and letter sounds. The second category 
of combined interventions included studies of training activities that combined PA instruction and 
instruction in some aspect of phonics or decoding. Often, this phonics training involved teaching 
children about letters and simple decoding tasks involving the use of letter sounds. There were also 
three studies that evaluated the effectiveness of alphabet instruction alone (all three of these studies in 
this category examined the impact of exposure to Sesame Street®-like video materials). 

In analyzing the results from these studies, the first question was whether these code-focused 
interventions were effective in promoting young childrens early literacy and conventional 
literacy skills across a broad range of outcome measures. These analyses were followed by analyses 
designed to answer questions concerning the relative impacts of different kinds of code-focused 
interventions and how these interventions worked with various developmental and demographic 
categories of children. Thus, these analyses reveal whether and how much code-focused 
interventions positively influence young children’s early literacy and conventional literacy skills, 
as well as whether various instructional or child characteristics moderate these overall impacts. 



Overall Estimates of Intervention Impacts 

The results related to the overall impacts of this group of intervention studies across all outcome 
variables are presented in Table 3.1. The data reported in the table include the outcome variables, 
the numbers of studies that contributed to the effect estimates, an estimate of the ESs based on a 
fixed-effect model, an estimate of the ESs based on a random-effect model,' the 95 percent upper 
and lower bound of the ESs based on the random-effect model, and the statistical significance of 
the ESs from the random-effect model. A large number of studies in this category examined the 
impacts of the interventions on outcome variables reflecting PA (51 studies), AK (24 studies), 
reading (36 studies), spelling (15 studies), and oral language (14 studies), fewer studies of these 
interventions examined the impacts on outcome variables reflecting general cognitive ability (two 
studies); memory (nine studies); print knowledge (five studies); rapid automatic naming (RAN) 
(eight studies); reading readiness (three studies); and writing (five studies). None of these studies 
considered the impact of the interventions on visual or perceptual processing as an outcome 
variable. It should be noted that, although specific tests of cognitive ability or memory per se 
were not identified in Chapter Two as being particular predictors of later literacy achievement, 
such measures are clearly implicated in various IQ tests, which were identified as significant 
predictors in Chapter Two. Therefore, the results of these measures are reported in these tables, 
since they were used in the various qualifying studies. It is possible that the pool of data on these 
unsearched-for variables would include more studies than were examined here. 



‘ Except where noted, all ES estimates and associated statistics ate based on a random-effect model. 
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Table 3. 1. Estimates of Effect Sizes Across Outcome Domains for Interventions Classified as Code-Focused 
for Each Dependent Variable 



Dependent Variable 


Fixed 

ES 


Random 

ES 


95% 

Cl 


Nof 

Studies 


pfor 

ES 








Lower 


Upper 












Bound 


Bound 






AK 


0.31 


0.38 


0.18 


0.58 


24 


0.0002 


Cognitive ability 


-0.47 


-0.41 


-0.78 


-0.01 


2 


0.04 


Memory 


0.20 


0.27 


0.06 


0.48 


9 


0.01 


Oral language 


0.27 


0.32 


0.09 


0.56 


14 


0.008 


PA 


0.76 


0.82 


0.68 


0.96 


51 


<0.0001 


Print knowledge 


0.44 


0.47 


0.18 


0.76 


5 


0.0013 


RAN 


0.35 


0.38 


0.08 


0.69 


8 


0.013 


Reading readiness 


0.20 


0.20 


0.02 


0.38 


3 


0.034 


Reading 


0.41 


0.44 


0.27 


0.60 


36 


<0.0001 


Spelling 


0.55 


0.61 


0.43 


0.80 


15 


<0.0001 


Writing 


0.43 


0.61 


0.18 


1.04 


5 


0.006 



As can be seen in Table 3.1, code-focused interventions usually had moderate to large effects both 
on measures of conventional literacy (i.e., reading, spelling) and on measures of precursor literacy 
skills (e.g., PA, AK). ESs of the interventions across all outcome variables were statistically reliable 
(i.e., p < 0.05). In all but one case, the average ESs for code-focused interventions were positive. 
Based on two studies, the impact of code-focused intervention on childrens cognitive skills just 
achieved significance, and the average effect was negative. However, this category of outcome 
variable had too few studies to allow unambiguous interpretation. Consequently, the results 
reported in Table 3.1 indicate that code-focused interventions have a significant, substantial, 
and positive impact both on young children’s conventional literacy skills and on early skills that 
predict later literacy achievement. 

The largest impact of code-focused interventions was on PA, with an average ES of 0.82. This 
result means that, on average, children who received a code-focused intervention scored 0.82 
of a standard deviation higher on measures of PA than did children who did not receive a code- 
focused intervention. To put this in context, if the average children not receiving a code-focused 
intervention scored 1 00 on a standardized test of PA that had a mean of 1 00 and a standard 
deviation of 15, the average children receiving a code-focused intervention scored 112 on the test 
(i.e., the difference between scoring at the 50th and 79th percentiles). 

The confidence intervals (CIs) reported in Table 3.1 can be used to compare the values of 
observed ESs across the outcome variables. The 95 percent Cl (i.e., upper and lower bounds of 
the ES) establishes the likely true value of the ES. That is, given the range of ESs observed across 
studies, the true value of the ES is likely to fall within the Cl 95 percent of the time. The Cl 
for an ES is affected by the variation in the ESs across studies as well as the number of studies 
included (e.g., a wider range of observed ESs and a smaller number of studies result in larger 
CIs). If the CIs do not overlap, the ESs are significantly different from each other at the 
p < 0.05 level. 
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The impact of code-focused interventions on PA was significantly larger than the impacts of 
code-focused interventions on oral language, AK, memory, reading readiness, and reading. The 
impacts of code-focused interventions on print knowledge, RAN, spelling, and writing were not 
statistically lower than their impact on PA. Finally, the impacts of code-focused interventions were 
statistically equal across conventional literacy outcome measures (i.e., reading, spelling, writing). 



Estimates of Intervention Impacts by Characteristics of Interventions 
and Populations 

In addition to the analyses of the overall effects of code-focused interventions on childrens 
literacy development, analyses were conducted that addressed questions about the effects of 
different types of code-focused interventions and on how code-focused interventions differed 
in their impacts on childrens skills across different population characteristics of the samples 
included in the studies. These estimations of effects by subcategory of intervention or population 
characteristic could be conducted only in cases in which sufficient numbers of studies used a 
specific outcome measure. Using a criterion of 1 0 or more studies^ that used a particular outcome 
measure, sufficient studies examined the impact of code-focused interventions on PA, AK, oral 
language, reading, and spelling to allow subanalyses to be conducted for these outcome domains. 

It is important to note that most of the code-focused interventions examined here were implemented 
using individual or small-group instruction or the studies were unclear about how the instruction 
was delivered. Because of this, it is not possible, on the basis of these analyses, to determine the 
effectiveness of whole-class or large-group code-focused instruction with young children. 

First, analyses were conducted to determine whether there were any differences in ES estimates 
based on the type of study design (e.g., RCT versus QED). A summary of the results for each 
of the five outcome variables is shown in Table 3.2. Complete results for these subanalyses are 
included in Appendix 3. A. ES estimates were similar for RCTs and QEDs for the PA outcomes 
(Q[l, 44] = 1.28,/) = 0.26), the oral language outcomes (Q[l, 10] = 0.42,/) = 0.51), the reading 
outcomes (Q[l, 29] = 0.16,/) = 0.69), and the spelling outcomes (Q[l, 12] = 1.31,/)= 0.25). The 
ES estimates for the AK outcomes were larger when the study design was RCT than when the study 
design was QED (Q[l, 19] = 5.76, p = 0.016). Eor the majority of outcome variables, type of study 
did not have a significant influence on the estimated impact of the code-focused interventions. 

Table 3.2. Summary of Effect Sizes for Outcome Variables for Study Classification Based on Type of Study 



ESs for Outcome Variable and (n) of Studies Contributing to ES 



Design of Study 


PA 


AK 


Oral Language 


Reading 


Spelling 


RCT 


0.87"* 


0.66*" 


0.25 


0.55*** 


0.45** 


(25) 


(12) 


(4) 


(15) 


(7) 


QED 


0.70*" 


0.16 


0.44* 


0.47*** 


0.64*** 


(21) 


(9) 


(8) 


(16) 


(7) 


Note: ESs based on random-effect model. 


p<0.05. **p<0.01, 


•**p< 0.001 









^ The limit of 1 0 studies was an arbitrary criterion. However, given that most planned subanalyses involved three or more 
categories, 10 was judged the number that would maximize the number of outcome variables included and minimize the 
subcategories with no studies for an analyzed outcome variable. 
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Analysis of Intervention Effects by Children's Age and Developmental Level 



The studies involving the impacts of code-focused interventions could have included a broad age 
range of children, but, in fact, few studies investigated the effects of these interventions with very 
young children. Most of the children in these studies were three, four, or five years old. However, 
it is possible that, for developmental or experiential reasons, the impact of these interventions 
differed for children who were preschool age and those who were already in kindergarten. To 
examine whether code-focused interventions had different effects depending on the age of the 
children in the study, where possible,^ studies were divided into two groups: one group of studies 
in which all of the children were not yet in kindergarten and one group of studies in which all of 
the children were in kindergarten. Studies with blended samples were excluded from the analyses 
(i.e., studies that combined the results of preschool- and kindergarten-age children in the same 
analyses were not used in this comparison). 

A summary of the estimates of ESs of code-focused interventions for preschool-age and 
kindergarten-age children separately is shown in Table 3.3. Complete results for these subanalyses 
are included in Appendix 3.B. There were no statistically significant differences in the ES estimates 
for PA (Q[l, 46] = 0.10, p = 0.75), AK (Q[l, 21] = 2.21, ^ = 0.14), oral language (Q[l, 12] = 

0.09, = 0.76), reading (Q[l, 32] = 1.48, />= 0.22), and spelling (Q[l, 13] = 0.55, />= 0.46). ESs 
were somewhat larger for studies that included preschool children than for those that included 
kindergarten children for AK, reading, and spelling outcomes; these differences were not statistically 
reliable. The separate ESs for preschool- and kindergarten-age children continued to be statistically 
reliable (except for the ES estimates for these interventions with oral language outcomes). 



Table 3.3. Summary of Effect Sizes for Outcome Variables for Study Classification Based on Age of Children 
in Study 





ESs for Outcome Variable and (n) of Studies Contributing to ES 


Age Group 


PA 


AK 


Oral Language 


Reading 


Spelling 


Preschool 


0.87“* 


0.67“ 


0.26 


0.75“ 


0.78“ 


(10) 


(5) 


(3) 


(4) 


(2) 


Kindergarten 


0.81"* 


0.32“ 


0.34* 


0.43*“ 


0.58*“ 


(38) 


(18) 


(11) 


(30) 


(13) 



Note: ESs based on random-effect model. *p<0.05, **p<0.01, ***p<0.001. 



A second set of subanalyses examined whether the estimated ESs varied as a function of the 
child’s developmental level in terms of literacy skills prior to intervention. When the studies 
provided information, the studies were classified as including children who (a) had little or 
no AK but were not yet readers, (b) had moderate to high AK but were not yet readers, or (c) 
were already readers. Studies that did not report children’s prior literacy knowledge or studies 
with blended samples (i.e., studies that included children with a broad range of prior literacy 
knowledge) were excluded from these subanalyses. The majority of the studies examining code- 
focused interventions either did not report this information for the sample or included children 
across a wide range of prior literacy knowledge. 

^ Because meta-analysis requires that a study contribute only one effect size to an analysis, studies that included samples that 
represented both preschool children and kindergarten children and that did not report separate analyses for preschool and 
kindergarten children could not be used in these analyses. 
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A summary of the results of these subanalyses is shown in Table 3.4. Complete results for these 
subanalyses are included in Appendix 3.C. ES estimates did not vary significantly as a function 
of children’s prior literacy level for the PA outcomes (Q[2, 19] = 1.43, p = 0.49), oral language 
outcomes (Q[l, 4] = \.7\,p = 0.10), reading outcomes (Q[2, 10] = 2.05, /> = 0.36), or spelling 
outcomes (Q[2, 5] = 0.49, p = 0.78). ES estimates were significantly different for AK outcomes 
across children’s prior literacy levels (Q[l, 8] = 6.32, p = 0.012) (see Tables 3.4 and 3.B.2). ESs 
were larger for children with little or no prior AK than they were for children who had already 
had moderate to high prior levels of AK. This difference is likely the result of a ceiling effect on 
measures of AK (i.e., there was less possibility to measure growth in AK for children who had 
high levels of AK prior to the intervention). 

Table 3.4. Summary of Effect Sizes for Outcome Variables for Study Classification Based on Prior Literacy 
Level of Study Sample 



ESs for Outcome Variable and (n) of Studies Contributing to ES 



Prior Literacy Level 


PA 


AK 


Oral Language 


Reading 


Spelling 


Little letter knowledge 


0.99'* 

(5) 


0.86" 

(4) 


0.07 

(3) 


0.92* 

(4) 


0.90 

(2) 


Letter knowledge but 


0.87™ 


0.09 


0.31 


0.38 


0.70* 


non reader 


(15) 


(5) 


(3) 


(7) 


(5) 


Reader 


1 .36** 


— 


— 


0.35 


0.53 


(2) 


(0) 


(0) 


(2) 


(1) 



Note: ESs based on random-effect model. *p<0.05, **p<0.01, ***p< 0.001. 



Overall, these two sets of subanalyses indicate that the strong, positive, and statistically 
significant impacts of code-focused interventions on children’s skills in the domains of PA, AK, 
oral language, reading, and spelling reported for the overall analyses hold regardless of the age 
of the children included in the studies and, for most outcomes, the prior literacy levels of the 
children included in the studies. These findings are important because they indicate (a) that it 
is possible to affect substantially those skills that are most predictive of later decoding, reading 
comprehension, and spelling for preschool-age children; (b) that these interventions show 
positive effects on reading and spelling skills (presumably mediated, in part, by the positive 
impacts on PA and AK); (c) that these results can be obtained with preschool- age children as well 
as with kindergarten children; and (d) that these substantial impacts are consistent regardless of 
children’s existing early literacy skills. 

Analysis of Intervention Effects, by Type of Intervention 

As noted previously, code-focused interventions were classified into four categories: (a) 
interventions that included PA instruction only, (b) interventions that included both PA 
instruction and AK instruction, (c) interventions that included AK instruction only, and (d) 
interventions that included both PA instruction and phonics instruction. Eor each of the five 
outcome variables with sufficient studies to allow subanalyses, ESs were computed separately 
for each of these four subtypes of code-focused intervention. The estimates of ESs from these 
analyses are shown in Tables 3.5 through 3.9. 
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Table 3.5. Impact of Code-Focused Interventions on Phonological Awareness Outcome by Nature of 
Intervention in Study 



Intervention 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








PA training only 


0.91 


0.12 


0.68 


1.15 


7.54 


21 


<0.00001 


PA and AK training 


0.70 


0.13 


0.45 


0.95 


5.43 


18 


<0.00001 


AK training only 


0.48 


0.49 


-0.47 


1.43 


0.99 


1 


0.3288 


PA and phonics training 


0.74 


0.13 


0.49 


0.99 


5.84 


19 


<0.00001 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.6. Impact of Code-Focused Interventions on Alphabet Knowledge Outcome by Nature of 
Intervention in Study 



Group 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








PA training only 


0.04 


0.11 


-0.19 


0.26 


0.31 


6 


0.76 


PA and AK training 


0.37 


0.13 


0.11 


0.62 


2.80 


7 


0.012 


PA and phonics training 


0.57 


0.12 


0.34 


0.81 


4.75 


9 


<0.0001 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison,, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3. 7. Impact of Code-Focused Interventions on Oral Language Outcome by Nature of Intervention 
in Study 



Group 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








PA training only 


0.09 


0.13 


-0.18 


0.35 


0.65 


4 


0.54 


PA and AK training 


0.13 


0.17 


-0.20 


0.46 


0.75 


4 


0.47 


AK training only 


0.83 


0.27 


0.30 


1.36 


3.05 


1 


<0.014 


PA and phonics training 


0.68 


0.17 


0.34 


1.02 


3.90 


4 


< 0.004 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.8. Impact of Code-Focused Interventions on Reading Outcome by Nature of Intervention in Study 



Intervention Type 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








PA training only 


0.19 


0.16 


-0.12 


0.50 


1.22 


10 


0.23 


PA and AK training 


0.31 


0.14 


0.04 


0.59 


2.22 


13 


0.033 


AK training only 


-0.52 


0.46 


-1.42 


0.37 


-1.15 


1 


0.26 


PA and phonics training 


0.66 


0.13 


0.41 


0.92 


5.08 


17 


0.00001 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison,, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Table 3.9. Impact of Code-Focused Interventions on Spelling Outcomes by Nature of Intervention in Study 



Intervention Type 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








PA training only 


0.59 


0.18 


0.23 


0.95 


3.20 


4 


0.006 


PA and AK training 


0.50 


0.14 


0.21 


0.78 


3.44 


6 


0.0037 


PA and phonics training 


0.59 


0.13 


0.34 


0.83 


4.64 


8 


0.0003 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability 



There were no significant differences between ESs of the different categories of code-focused 
intervention for the PA outcome variable (Table 3.5) (Q[3, 55] = 2.07, p = 0.56) or the spelling 
outcome variable (Table 3.9) (Q[2, 15] = 0.24, p = 0.89). ESs did differ significantly between 
subcategories of code-focused interventions for the AK outcome variable (Table 3.6) (Q[2, 

19] = 10.76, p = 0.005), the oral language outcome variable (Table 3.7) (Q[3, 9] = 12.05, p = 
0.007), and the reading outcome variable (Table 3.8) (Q[3, 37] = 10.27, p = 0.02). Eor the AK 
outcome, the ES estimate for the PA training-only condition was significantly smaller than that 
for PA and phonics training, suggesting a degree of specificity between the knowledge and skills 
targeted by the intervention and the measured impact on children’s skills. Eor the oral language 
and reading outcome variables, the ES estimate from the single study for AK training only was 
either substantially higher or substantially lower than those from the other forms of intervention. 
However, because only one study contributed to the ES estimate for the effect of AK training, 
these results are not interpretable. 

The results of these analyses indicate that the impacts of most code-focused interventions are 
positive, moderate to large, and statistically reliable across a broad range of key early literacy and 
reading indicators (i.e., PA, AK, reading, spelling). Not surprisingly, the interventions that did 
not include a print-focused component (i.e., those with PA training only) had a significantly 
weaker effect on print-specific outcomes (i.e., AK). Regardless, the results were generally 
consistent across outcome domains, indicating that interventions that include variations of PA 
training affect not only PA skills but also measures of reading and spelling. 

In addition to examining the relative impacts of different types of code-focused interventions, 
the relative impacts of variations in the nature of the PA interventions were examined. PA varies 
along at least two independent dimensions: level of linguistic complexity and cognitive operation. 
Level of linguistic complexity refers to the size of the sound unit on which PA is demonstrated, 
and it ranges along a continuum from word-level units to phoneme-level units. The target skill 
of different PA interventions is sometimes one point on this continuum and sometimes multiple 
levels of this continuum. A common theoretically relevant split on this continuum is phoneme- 
level tasks or targets (i.e., phonemic awareness) versus subphonemic tasks or targets (i.e., syllable 
awareness, onset-rime awareness). Cognitive operation refers to the type of task performed on these 
linguistic units and can involve identity (e.g., rhyme oddity detection), synthesis (e.g., blending 
or putting linguistic units together to form new linguistic units, typically words), or analysis 
(e.g., separating a linguistic unit from a larger linguistic unit through deletion or counting), with 
analysis tasks often considered the more developmentally advanced cognitive operation. 
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Studies that provided the necessary information were classified as including PA interventions that 
involved a focus on (a) phonemic awareness, (b) subphonemic awareness, or (c) both phonemic 
and subphonemic awareness. The studies also were classified as including interventions that 
focused on (a) analysis skills, (b) synthesis skills, or (c) both analysis and synthesis skills. Few or 
no interventions focused solely on identity skills, and therefore, identity-focused PA intervention 
was not a category that was analyzed. Studies that did not report sufficient details about the 
nature of the PA intervention in terms of either linguistic complexity or cognitive operation were 
excluded from these analyses. More than half of the studies examining code-focused interventions 
involving a component of PA training could be classified for level of linguistic complexity or 
cognitive operation. 

A summary of the results of the analyses for level of linguistic complexity targeted by the PA 
intervention is shown in Table 3.10. Complete results for these analyses are included in Appendix 
3.D. ES estimates did not vary significantly as a function of level of linguistic complexity targeted 
by the intervention for PA outcomes (Q[2, 37] = 4.19, p = 0.12), AK outcomes (Q[2, 14] = 

3.26, p = 0.20), oral language outcomes (Q[2, 7] = 2.81, = 0.25), reading outcomes (Q[2, 

25] = 0.96, p = 0.62), or spelling outcomes (Q[2, 11] = 0.54, p = 0.76). That is, regardless of 
whether the nature of the PA training involved phonemic-level tasks, subphonemic-level tasks, 
or a combination of phonemic- and subphonemic-level tasks, the estimated ESs were statistically 
similar, suggesting that variations along the continuum of linguistic complexity yield comparable 
positive effects on key early literacy and literacy measures for this age group of children. 

Table 3. 10. Summary of Effect Sizes for Outcome Variables for Study Classification Based on Focus of 
Pbonological Awareness Training 



ESs for Outcome Variable and (n) of Studies Contributing to ES 



Focus of PA Training 


PA 


AK 


Oral Language 


Reading 


Spelling 


Subphonemic 


1.10*** 

(14) 


0.62** 

(7) 


0.64* 

(3) 


0.41 

(8) 


0.76** 

(3) 


Phonemic 


0.71*** 


0.14 


0.23 


0.67** 


0.71* 


(7) 


(1) 


(1) 


(7) 


(2) 


Both 


0.75*** 


0.21 


0.14 


0.45** 


0.58*** 


(19) 


(9) 


(6) 


(13) 


(9) 


Note: ESs based on random-effect model. *p<0.05, **p<0.01, 


***p <0.001 









A summary of the results of the analyses for the cognitive operation targeted by the PA 
intervention is shown in Table 3.1 1. Complete results for these analyses are included in Appendix 
3.E. ES estimates did not vary significantly as a function of the type of cognitive operation 
targeted by the intervention for PA outcomes (Q[2, 33] = 0.20, p = 0.91), AK outcomes (Q[l, 

10] = 0.55, p = 0.46), oral language outcomes (Q[l, 6] = 0.23, p = 0.63), reading outcomes (Q[2, 
19] = 2.95, p = 0.23), or spelling outcomes (Q[2, 10] = 1.84, p = 0.40). As for the analyses for 
type of cognitive operation targeted by PA interventions, regardless of whether the nature of the 
PA training involved analysis tasks, synthesis tasks, or a combination of analyses and synthesis 
tasks, the estimated ESs were statistically similar, suggesting that variations in the type of 
cognitive operation used in PA training yield comparable positive effects on key early literacy and 
literacy measures for this age group of children. 
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Table 3. 1 1. Summary of Effect Sizes for Outcome Variables for Study Classification Based on Nature of 
Pbonoiogicai Awareness Training 



ESs for Outcome Variable and (n) of Studies Contributing to ES 



Type of PA Training 


PA 


AK 


Oral Language 


Reading 


Spelling 


Analysis 


0.88*** 

(8) 


0.13 

(3) 


0.13 

(1) 


0.13 

(5) 


0.53** 

(4) 


Synthesis 


0.72* 

(5) 


(0) 


(0) 


0.45 

(2) 


-0.16 

(1) 


Both 


0.83*** 

(23) 


0.34* 

(9) 


0.39 

(7) 


0.49*** 

(15) 


0.56*** 

(8) 


Note: ESs based on random-effect model. 


*p<0.05, 


**p<0.01, ***p <0.001 









Analysis of Intervention Effects by Demographics of Study Samples 

For all analyses of intervention effects, NELP examined variations in estimated ESs for a 
common set of demographic variables. The demographic variables in these analyses included the 
socioeconomic status (SES) of the families of the children included in the study samples, ethnic 
classification of study participants, and the population density of the location where the study 
was conducted (i.e., rural, urban, suburban, mixed, unknown). These analyses allow both an 
examination of reliable systematic variation across categories of these demographic variables and 
a summary of the texture of the literature regarding code-focused interventions for children in 
preschool or kindergarten. In the majority of studies included in the summary of code-focused 
interventions, either these demographic characteristics were not reported or the samples used in 
the studies represented a mix of the demographic categories. 

A summary of the results of the analyses for ESs based on the SES of the study samples in code- 
focused interventions is shown in Table 3.12. Complete results for these analyses are included 
in Appendix 3.E ES estimates did not vary significantly as a function of the SES classification of 
the study samples for the AK outcomes (Q[2, 9] = 0.06, p = 0.97), oral language outcomes (Q[2, 
5] = 0.18, /) = 0.92), or reading outcomes (Q[2, 7] = 0.40, p = 0.82). ES estimates did vary as a 
function of SES for the PA outcomes (Q[2, 16] = 7.30, p < 0.03) and spelling outcomes (Q[l, 4] 

= 5.35, p = 0.02). CIs shown in Table 3.E.1 indicate that, for PA outcomes, studies with samples 
classified as mixed SES had smaller ESs than did studies with samples classified as not-low SES. 
CIs shown in Table 3.E.5 indicate that, for spelling outcomes, studies with samples classified as 
low SES had smaller ESs than did studies with samples classified as not-low SES. In both cases 
of statistically significant variation in ESs, however, it is likely that the small number of studies 
classified as mixed SES (PA outcomes) or low SES (spelling outcomes) contributed to this finding 
and should, therefore, not be given substantial interpretive credence. 

A summary of the results of the analyses for ESs of code-focused interventions based on the 
ethnicity of the study samples is shown in Table 3.13. Complete results for these analyses are 
included in Appendix 3.G. ES estimates did not vary significantly as a function of the ethnicity 
classification of the study samples for the PA outcomes (Q[2, 45] = 2.81, = 0.25), AK outcomes 

(Q[l,21] = 0.10, = 0.75), oral language outcomes (Q[l, 12] = 2.02, ^ = 0.16), or reading 

outcomes (Q[l, 32] = 0.05, p = 0.82). All studies with spelling outcomes were classified as mixed 
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ethnicity or unknown. It is important to note that the majority of studies included in these 
analyses had samples that were classified as mixed or unknown ethnicity. Consequently, the 
absence of statistically reliable variability in ESs dependent on the ethnicity of the children has 
not been adequately tested. Based on existing studies, however, there is no evidence on which to 
conclude that childrens ethnicity moderates the positive effects of code-focused interventions. 

Table 3. 12. Summary of Effect Sizes for Outcome Variables for Study Classification Based on 
Socioeconomic Status of Study Sample 



ESs for Outcome Variable and (n) of Studies Contributing to ES 



SES Classification 


PA 


AK 


Oral Language 


Reading 


Spelling 


Low 


0.81*" 


0.40 


0.26 


0.60 


0.76" 


(9) 


(6) 


(5) 


(5) 


(5) 


Not low 


1.42*" 


0.49 


0.19 


0.26 


2.00* 


(8) 


(5) 


(2) 


(3) 


(1) 


Mixed 


0.40 


0.33 


0.39 


0.57 




(2) 


(1) 


(1) 


(2) 


(0) 


Note: ESs based on random-effect model. 


*p<0.05, **p<0.0l, 


•**p< 0.001. 









Table 3. 13. Summary of Effect Sizes for Outcome Variables for Study Classification Based on Ethnicity of 
Study Sample 



ESs for Outcome Variable and [n] of Studies Contributing to ES 



Ethnicity 


PA 


AK 


Oral Language 


Reading 


Spelling 


Caucasian 


1 49" 


0 52 




0 35 




(2) 


(2) 


(0) 


(1) 


(0) 


African American 


0.41 


— 


— 


— 


— 


(1) 


(0) 


(0) 


(0) 


(0) 








0 83* 






Hispanic 


(0) 


(0) 


(1) 


(0) 


(0) 


Mixed/unspecified 


0 81*** 


0 39" 


0 27* 


0 47*** 


0 61*** 


(45) 


(21) 


(13) 


(33) 


(15) 



Note: ESs based on random-effect model. *p<0.05, **p<0.01, ***p<0.00l. 



A summary of the results of the analyses of ESs for code-focused interventions based on 
population density is shown in Table 3.14. Complete results for these analyses are included in 
Appendix 3.H. ES estimates did not vary significantly as a function of population density for 
PA outcomes (Q[4, 42] = 2.24, p = 0.69), AK outcomes (Q[3, 19] = 64.78, p < 0.0001), oral 
language outcomes (Q[4, 9] = 8,49, p = 0.08), reading outcomes (Q[4, 27] = 1.90, p = 0.76), or 
spelling outcomes (Q[3, 10] = 5.56, p = 0.14). Again, most studies were classified as unknown 
population density. Consequently, the absence of statistically reliable variability in ESs dependent 
on population density has not been adequately tested. Based on existing studies, however, there is 
no evidence on which to conclude that population density moderates the positive effects of code- 
focused interventions. 
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Table 3. 14. Summary of effect sizes for outcome variables for study classification based on population 
density of study sample 





Effect Sizes for Outcome Variable and (n) of Studies Contributing to Effect Size 


Population Density 


Phonological 

Awareness 


Alphabet 

Knowledge 


Oral Language 


Reading 


Spelling 


Rural 


.60" 


.01 


.08 


.13 


___ 


(4) 


(3) 


(2) 


(2) 


(0) 


Urban 


.93"' 


.29" 


.81" 


.48** 


.58*" 


(17) 


(8) 


(3) 


(11) 


(8) 


Suburban 


90* 




30 


46 


71* 


(2) 


(0) 


(3) 


(2) 


(1) 


Mixed 


.92"* 


.27 


.05 


.64* 


1.19" 


(3) 


(2) 


(2) 


(3) 


(1) 


Unknown 


.74*** 


Ql-J*** 


.40 


.56*" 


.46** 


(21) 


(10) 


(4) 


(14) 


(4) 



Note. Effect sizes based on random-effect model. *p<.05, **p<.Ot, ***p<.001. 



Summary and Conclusions 

Results from this meta- analysis of the impacts of code-focused interventions on the early 
literacy and conventional literacy skills of young children indicate that these interventions yield 
a moderate to large effect on the predictors of later reading and writing (i.e., PA, AK) and on 
measures of reading and writing. These effects were robust to variation in the type of code- 
focused intervention, to variation in childrens ages or developmental levels, and to variations 
in methods of teaching young children PA. At this time, few studies allow fine-grained analysis 
of other population variables, such as SES, ethnicity, or population density. However, existing 
studies provide no evidence that the effects of code-focused interventions are altered by these 
sample characteristics. 

The majority of code-focused interventions involved some form of PA training activity. 
Consequently, most of the substantially positive impacts on childrens early literacy skills need to 
be interpreted in this context. That is, these analyses show that some form of PA training, either 
alone or in combination with more or less complex instruction related to print knowledge (i.e., 
letter-name instruction, instruction in early decoding skills) is likely to yield growth in children’s 
skills related to later reading and writing achievement. Whereas the literature contains both 
debate and findings concerning the type of PA training required to produce positive impacts 
on reading skills, the results of these analyses did not reveal any statistically reliable differences 
between variations in PA interventions. Categorizing the nature of PA training according to two 
theoretically relevant dimensions, the level of linguistic complexity that was the focus of the 
training and the nature of the cognitive operation taught in the PA training, did not indicate that 
one form of training was more or less effective than another form of training across a range of 
outcome measures. 

Importantly, there was no evidence that the effectiveness of code-focused interventions was 
influenced by age or developmental level of the children. That is, the impacts of code-focused 
interventions were observed in children whether they were preschool age or kindergarten age. 
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and these interventions were equally successful across a range of levels of prior literacy knowledge 
(from minimal AK to being able to read). These findings indicate that there is not a point along 
either an age or a developmental continuum at which code-focused interventions become more 
or less beneficial to childrens early literacy skills. The findings also suggest that there is no 
preexisting level of knowledge or skill that children must attain before these interventions can be 
used successfully. 

Most of the code-based interventions tested here are not available commercially. The majority of 
interventions included in these analyses were designed and implemented by researchers, and there 
was a great deal of variability in the specifics of the various interventions. This suggests that some 
instructional variations may be more effective than others, so, ultimately, it will be important 
and necessary to distill the specific components of these interventions to determine what types of 
intervention activities produce the most positive effects on childrens early literacy skills. It is not 
sufficient to merely label interventions as PA training, phonics, or code focused for them to be 
effective. Successful code-focused interventions will likely include all or most of the components 
of the interventions noted in this meta- analysis; thus, interventions should include PA training 
with activities involving higher-level PA skills, such as actively engaging in analysis or synthesis 
of words at the syllable, onset-rime, or phoneme level with feedback on correct and incorrect 
responses. Although PA training can be conducted alone, the results of this meta-analysis suggest 
that there may be an advantage of combining such training with activities designed to teach 
children about specific aspects of print, such as letter names and letter sounds. 

The majority of the code-focused interventions summarized by this meta-analysis were 
conducted as either individual-level or small group-level interventions. There was no evidence 
that whole-class or large-group code-focused interventions will produce similar-sized effects 
on childrens reading-related skills. While it is not the case that research has shown whole-class 
or large-group implementation of code instruction to be ineffective (such approaches were 
not tested at all), it would be a mistake to assume that teachers could successfully implement 
these interventions with large groups. Future research may be able to find how to make such 
instruction effective when delivered to whole classes or large groups, but, until such work is 
done, it would be prudent to deliver such instruction in ways more similar to those that were 
successful. It is likely that, for some of the interventions, a small-group format will be necessary 
to achieve this level of positive results. 

Finally, in addition to providing information about the overall impacts of code-focused 
interventions as well as the robustness of these interventions across different outcome 
variables, variations in type of intervention, and populations of children, these analyses provide 
information on those things that have not yet been studied or have not been studied well to date. 
Interestingly, we found only one study that investigated the effect of teaching letter knowledge 
to children. Direct teaching of the alphabet was most generally examined in the context of some 
form of PA training. Consequently, there is, at present, no interpretable evidence that teaching 
AK alone is a sufficient technique for enhancing childrens reading-related skills. 

In some cases, a meta- analysis of existing studies could not be used to address important 
questions that early childhood educators face. Most notably, extant studies do not allow an 
adequate examination of the relative effectiveness of code-focused instruction for specific 
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subpopulations of children. To their credit, most studies included mixed samples of children 
from different socioeconomic backgrounds, ethnic groups, and living environments (e.g., 
population density). Unfortunately, the data in these studies were usually not reported in a 
way that differential effectiveness could be studied. Although the early childhood education 
field is interested in specific questions about which interventions will work best for children 
living in poverty, children from traditionally underrepresented ethnic groups, children who are 
English-language learners, or children growing up in rural or urban environments, there are 
not yet studies focusing on these specific subpopulations or that allow examination of these 
subpopulations to answer these questions. Given the clear success of code-focused instruction 
with these mixed populations, it seems prudent to make such instruction available to all 
populations of young children, at least until research more directly addresses this question. 
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Appendix 3.A: Summary of Results for Subanalyses Based on the Type of 
Study Design: Randomized Controlled Trial Versus Quasiexperimental Study 

Table 3.A. 1. Subanalysis for Pbonological Awareness Outcomes by Study Design 



Type of Study Mean ES 


SE 


95% Cl 


t 


P 


n 






Lower 

Bound 


Upper 

Bound 








RCT 0.87 


0.10 


0.67 


1.07 


8.44 


<0.00001 


25 


QED 0.70 


0.11 


0.49 


0.91 


6.59 


<0.00001 


21 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability 


■ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.A.2. Subanalysis for Alphabet Knowledge Outcomes by Study Design 








Type of Study Mean ES 


SE 


95% Cl 


t 


p 


n 






Lower 

Bound 


Upper 

Bound 








RCT 0.66 


0.14 


0.38 


0.94 


4.59 


0.0002 


12 


QED 0.16 


0.15 


-0.12 


0.45 


1.12 


0.28 


9 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability 



Table 3.A.3. Subanalysis for Oral Language Outcomes by Study Design 



Type of Study Mean ES 


SE 


95% Cl 


t 


P 


n 






Lower 

Bound 


Upper 

Bound 








RCT 0.25 


0.24 


-0.22 


0.71 


1.02 


0.33 


4 


QED 0.44 


0.17 


0.11 


0.76 


2.62 


0.03 


8 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability 


^ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.A.4. Subanalysis for Reading Outcomes by Study Design 










Type of Study Mean ES 


SE 


95% Cl 


t 


P 


n 






Lower 

Bound 


Upper 

Bound 








RCT 0.55 


0.14 


0.27 


0.82 


3.82 


<0.0007 


15 


QED 0.47 


0.13 


0.21 


0.72 


3.60 


<0.0012 


16 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability 
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Table 3.A.5. Subanalysis for Spelling Outcomes by Study Design 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








RCT 


0.45 


0.12 


0.21 


0.69 


3.72 


< 0.003 


1 


QED 


0.64 


0.11 


0.43 


0.84 


6.05 


<0.0001 


1 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability 



Appendix 3.B: Summary of Results for Subanalyses Based on Age of 
Children Included in Study Sample 



Table 3.B.I. Impact of Code-Focused Interventions on Pbonological Awareness Outcomes by Age of 
Cbildren in Study 



Age Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Preschool 


0.87 


0.18 


0.52 


1.22 


4.83 


0.00002 


10 


Kindergarten 


0.81 


0.08 


0.64 


0.97 


9.49 


<0.00001 


38 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.B.2. Impact of Code-Focused Interventions on Alphabet Knowledge Outcomes by Age of Cbildren 
in Study 



Age Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Preschool 


0.67 


0.21 


0.26 


1.07 


3.25 


0.004 


5 


Kindergarten 


0.32 


0.11 


0.12 


0.53 


3.07 


0.006 


18 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability. 


■ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.B.3. Impact of Code-Focused Interventions on 


Oral Language Outcomes by Age of Cbildren in Study 


Age Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Preschool 


0.26 


0.25 


-0.23 


0.74 


1.02 


0.33 


3 


Kindergarten 


0.34 


0.14 


0.08 


0.61 


2.52 


0.03 


11 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Table 3.B.4. Impact of Code-Focused Interventions on Reading Outcomes by Age of Cbildren in Study 



Age Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Preschool 


0.75 


0.25 


0.26 


1.24 


3.02 


< 0.005 


4 


Kindergarten 


0.43 


0.09 


0.25 


0.61 


4.61 


<0.0001 


30 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability. 


: t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.B.5. Impact of Code-Focused Interventions 


on Spelling Outcomes by Age of Cbildren in Study 


Age Group 


Mean ES 


SE 


95% Cl 


t 


p 


n 








Lower 

Bound 


Upper 

Bound 








Preschool 


0.78 


0.25 


0.29 


1.27 


3.14 


< 0.008 


2 


Kindergarten 


0.58 


0.10 


0.38 


0.79 


5.62 


<0.0001 


13 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Appendix 3.C: Summary of Results for Subanalyses Based on Prior Level of 
Literacy Knowledge of Children Included in Study Sample 



Table 3.C. 1. Subanalysis for Pbonological Awareness Outcomes by Cbildren s Prior Level of Literacy Skill 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 


Upper 














Bound 


Bound 








Little letter knowledge 


0.99 


0.27 


0.46 


1.52 


3.67 


< 0.002 


5 


Letter knowledge but 
nonreader 


0.87 


0.15 


0.57 


1.16 


5.76 


<0.0001 


15 


Reader 


1.36 


0.39 


0.60 


2.12 


3.49 


<0.003 


2 


Note: ES = ES from random-effect model, SE = SE of ES, t = 


■ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


statistical probability. 
















Table 3.C.2. Subanalysis for Alphabet Knowledge Outcomes by Cbildren s Prior Level of Literacy Skill 


Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 


Upper 














Bound 


Bound 








Little letter knowledge 


0.86 


0.25 


0.38 


1.34 


3.51 


< 0.008 


4 


Letter knowledge but 
nonreader 


0.09 


0.19 


-0.27 


0.45 


0.48 


0.65 


6 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Table 3.C.3. Subanalysis for Oral Language Outcomes by Cbildren's Prior Level of Literacy Skill 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 


Upper 














Bound 


Bound 








Little letter knowledge 


0.07 


0.13 


-0.19 


0.32 


0.51 


0.64 


3 


Letter knowledge but 
nonreader 


0.31 


0.14 


0.04 


0.59 


2.25 


0.09 


3 


Note: ES = ES from random-effect model, SE = SE of ES, t = 


t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


statistical probability. 
















Table 3.C.4. Subanalysis for Reading Outcomes by Cbildren’s Prior Level of Literacy Skill 






Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 


Upper 














Bound 


Bound 








Little letter knowledge 


0.92 


0.32 


0.29 


1.55 


2.86 <0.02 


4 


Letter knowledge but 
nonreader 


0.38 


0.23 


-0.07 


0.83 


1.65 


0.13 


7 


Reader 


0.35 


0.41 


-0.46 


1.16 


0.85 


0.41 


2 


Note: ES = ES from random-effect model, SE = SE of ES, t = 


■ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


statistical probability. 
















Table 3.C.5. Subanalysis for Spelling Outcomes by Cbildren’s Prior Level of Literacy Skill 






Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 


Upper 














Bound 


Bound 








Little letter knowledge 


0.90 


0.35 


0.21 


1.60 


2.55 


0.05 


2 


Letter knowledge but 
nonreader 


0.70 


0.21 


0.28 


1.11 


3.32 


0.02 


5 


Reader 


0.53 


0.40 


-0.26 


1.32 


1.32 


0.25 


1 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Appendix 3.D: Summary of Results for Subanalyses Based on Classification 
of Studies for Level of Linguistic Complexity Targeted by the Phonological 
Awareness Intervention 



Table 3.0. 1. Subanalysis for Pbonological Awareness Outcomes by Level of Linguistic Complexity Targeted 
by Pbonological Awareness Intervention 



Linguistic Complexity 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Subphonemic 


1.10 


0.15 


0.81 


1.39 


7.45 


<0.00001 


14 


Phonemic 


0.71 


0.20 


0.33 


1.10 


3.63 


< 0.0009 


7 


Both 


0.75 


0.12 


0.52 


0.97 


6.45 


<0.00001 


19 



Note: ES = ES from random-effect model, SE=SE ofES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate., p = 
statistical probability. 
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Table 3.D.2. Subanalysis for Alphabet Knowledge Outcomes by Level of Linguistic Complexity Targeted by 
Phonological Awareness Intervention 



Linguistic Complexity 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Subphonemic 


0.62 


0.19 


0.24 


1.00 


3.22 


< 0.007 


1 


Phonemic 


0.14 


0.39 


-0.62 


0.90 


0.35 


0.73 


1 


Both 


0.21 


0.14 


-0.05 


0.48 


1.56 


0.14 


9 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.D.3. Subanalysis for Oral Language Outcomes by Level of Linguistic Complexity Targeted by 
Phonological Awareness Intervention 



Linguistic Complexity 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Subphonemic 


0.64 


0.25 


0.14 


1.13 


2.53 


0.04 


3 


Phonemic 


0.23 


0.39 


-0.53 


1.00 


0.60 


0.57 


1 


Both 


0.14 


0.16 


-0.16 


0.45 


0.92 


0.39 


6 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.0.4. Subanalysis for Reading Outcomes by Level of Linguistic Complexity Targeted by Phonological 
Awareness Intervention 



Linguistic Complexity 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Subphonemic 


0.41 


0.21 


0.00 


0.82 


1.97 


0.06 


8 


Phonemic 


0.67 


0.21 


0.26 


1.09 


3.18 


< 0.004 


7 


Both 


0.45 


0.16 


0.14 


0.75 


2.87 


< 0.009 


13 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.0.5. Subanalysis for Spelling Outcomes by Level of Linguistic Complexity Targeted by Phonological 
Awareness Intervention 



Linguistic Complexity 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Subphonemic 


0.76 


0.23 


0.31 


1.22 


3.27 


<0.008 


3 


Phonemic 


0.71 


0.27 


0.19 


1.23 


2.69 


0.02 


2 


Both 


0.58 


0.13 


0.33 


0.84 


4.47 


<0.001 


9 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability 
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Appendix 3.E: Summary of Results for Subanalyses Based on Classification 
of Studies for Cognitive Operation Targeted by the Phonological Awareness 
Intervention 



Table 3.E. 1. Subanalysis for Pbonological Awareness Outcomes by Cognitive Operation Targeted by 
Pbonological Awareness Intervention 



Operation 


Mean ES 


SE 


95%CI 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Analysis 


0.88 


0.19 


0.50 


1.26 


4.51 


<0.0001 


8 


Synthesis 


0.72 


0.29 


0.16 


1.28 


2.53 


<0.02 


5 


Both 


0.83 


0.11 


0.61 


1.04 


7.49 


<0.00001 


23 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.E.2. Subanalysis for Alphabet Knowledge Outcomes by Cognitive Operation Targeted by 
Phonological Awareness Intervention 



Operation 


Mean ES 


SE 


95%CI 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Analysis 


0.13 


0.24 


1 

CD 

CO 

cn 


0.61 


0.54 


0.60 


3 


Both 


0.34 


0.13 


0.09 


0.58 


2.67 


<0.03 


9 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.E.3. Subanalysis for Oral Language Outcomes by Cognitive Operation Targeted by Phonological 
Awareness Intervention 



Operation 


Mean ES 


SE 


95%CI 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Analysis 


0.13 


0.51 


-0.87 


1.13 


0.25 


0.81 


1 


Both 


0.39 


0.18 


0.04 


0.74 


2.20 


0.07 


7 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.E.4. Subanalysis for Reading Outcomes by Cognitive Operation Targeted by Phonological 
Awareness Intervention 



Operation 


Mean ES 


SE 


95%CI 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Analysis 


0.13 


0.18 


-0.22 


0.49 


0.74 


0.47 


5 


Synthesis 


0.45 


0.34 


-0.23 


1.12 


1.30 


0.21 


2 


Both 


0.49 


0.11 


0.28 


0.71 


4.57 


<0.001 


15 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Table 3.E.5. Subanalysis for Spelling Outcomes by Cognitive Operation Targeted by Pbonological 
Awareness Intervention 



Operation 


Mean ES 


SE 


95%CI 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Analysis 


0.53 


0.17 


0.19 


0.87 


3.05 


<0.02 


4 


Synthesis 


-0.16 


0.53 


-1.19 


0.87 


-0.30 


0.77 


1 


Both 


0.56 


0.09 


0.38 


0.75 


6.00 


<0.001 


8 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Appendix 3F: Summary of Results for Subanalyses Based on Classification 
of Studies by Socioeconomic-Status Classification of Study Sample 



Table 3.F. 1. Subanalysis for Pbonological Awareness Outcomes by Socioeconomic-Status Classification of 
Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Low SES 


0.81 


0.18 


0.46 


1.17 


4.52 


<0.001 


9 


Not low SES 


1.42 


0.20 


1.02 


1.82 


6.96 


<0.00001 


8 


Mixed SES 


0.40 


0.42 


-0.42 


1.22 


0.95 


0.36 


2 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.F.2. Subanalysis for Alphabet Knowledge Outcomes by Socioeconomic-Status Classification of 
Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Low SES 


0.40 


0.26 


-0.11 


0.91 


1.55 


0.16 


6 


Not low SES 


0.49 


0.29 


-0.08 


1.05 


1.68 


0.13 


5 


Mixed SES 


0.33 


0.68 


-1.01 


1.67 


0.48 


0.64 


1 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.F3. Subanalysis for Oral Language Outcomes by Socioeconomic-Status Classification of Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Low SES 


0.26 


0.15 


-0.03 


0.55 


1.77 


0.14 


5 


Not low SES 


0.19 


0.23 


-0.26 


0.63 


0.81 


0.45 


2 


Mixed SES 


0.39 


0.45 


-0.50 


1.28 


0.85 


0.43 


1 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Table 3.F.4. Subanalysis for Reading Outcomes by Socioeconomic-Status Classification of Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


p n 








Lower 

Bound 


Upper 

Bound 






Low SES 


0.60 


0.33 


-0.04 


1.24 


1.85 


0.11 5 


Not low SES 


0.26 


0.45 


-0.64 


1.15 


0.56 


0.59 3 


Mixed SES 


0.57 


0.55 


-0.50 


1.65 


1.04 


0.33 2 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability. 


^ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.F5. Subanalysis for Spelling Outcomes by Socioeconomic-Status Classibcation of Study Sample 


Group 


Mean ES 


SE 


95% Cl 


t 


p n 








Lower 

Bound 


Upper 

Bound 






Low SES 


0.76 


0.16 


0.44 


1.08 


4.60 


0.01 5 


Not low SES 


2.00 


0.51 


1.00 


3.00 


3.92 


0.02 1 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Appendix 3.G: Summary of Results for Subanalyses Based on Classification 
of Studies by Ethnicity of Study Sample 



Table 3.G.I. Subanalysis for Pbonological Awareness Outcomes by Ethnicity of Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Caucasian 


1.49 


0.46 


0.60 


2.38 


3.27 


0.002 


2 


African American 


0.41 


0.52 


-0.61 


1.43 


0.78 


0.44 


1 


Mixed or unspecified 


0.81 


0.08 


0.65 


0.96 


10.43 


<0.00001 


45 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.G.2. Subanalysis for Alphabet Knowledge Outcomes by Ethnicity of Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Caucasian 


0.52 


0.37 


-0.22 


1.25 


1.38 


0.18 


2 


Mixed or unspecified 


0.39 


0.11 


0.18 


0.60 


3.69 


<0.002 


21 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Table 3.G.3. Subanalysis for Oral Language Outcomes by Ethnicity of Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Latino or Hispanic 


0.83 


0.37 


0.10 


1.55 


2.23 


0.05 


1 


Mixed or unspecified 


0.27 


0.12 


0.05 


0.50 


2.38 


0.04 


13 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability. 


■ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.G.4. Subanalysis for Reading Outcomes by Ethnicity of Study Sample 








Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Caucasian 


0.35 


0.51 


-0.64 


1.35 


0.70 


0.49 


1 


Mixed/unspecified 


0.47 


0.09 


0.29 


0.65 


5.13 


<0.0001 


33 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Table 3.G.5. Subanalysis for Spelling Outcomes by Ethnicity of Study Sample 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Mixed or unspecified 


0.62 


0.09 


0.43 


0.79 


6.76 


<0.0001 


15 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 



Appendix 3.H: Summary of Results for Subanalyses Based on Classification 
of Studies by Population Density of Study Location 



Table 3.H. 1. Subanalysis for Phonological Awareness Outcomes by Population Oensity of Study Location 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Rural 


0.60 


0.23 


0.15 


1.05 


2.60 


0.01 


4 


Urban 


0.93 


0.13 


0.67 


1.19 


7.03 


<0.00001 


17 


Suburban 


0.90 


0.37 


0.17 


1.62 


2.42 


<0.02 


2 


Mixed 


0.92 


0.27 


0.40 


1.45 


3.43 


0.001 


3 


Not specified 


0.74 


0.12 


0.51 


0.97 


6.34 


<0.00001 


21 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Table 3.H.2. Subanalysis for Alphabet Knowledge Outcomes by Population Density of Study Location 



Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Urban 


0.01 


0.06 


-0.11 


0.14 


0.23 


0.82 


3 


Rural 


0.29 


0.10 


0.09 


0.48 


2.85 


0.01 


8 


Mixed 


0.27 


0.15 


-0.03 


0.57 


1.79 


0.09 


2 


Not specified 


0.82 


0.08 


0.67 


0.97 


10.47 


<0.001 


10 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability. 


■ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.H.3. Subanalysis for Oral Language Outcomes by Population Density of Study Location 




Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Rural 


0.08 


0.18 


-0.28 


0.44 


0.42 


0.69 


2 


Urban 


0.81 


0.21 


0.39 


1.23 


3.77 


< 0.005 


3 


Suburban 


0.30 


0.19 


-0.07 


0.68 


1.59 


0.15 


3 


Mixed 


0.05 


0.23 


-0.39 


0.50 


0.24 


0.82 


2 


Not specified 


0.40 


0.18 


0.04 


0.75 


2.20 


0.06 


4 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability. 


t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.H.4. Subanalysis for Reading Outcomes by Population Density of Study Location 




Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Rural 


0.13 


0.31 


-0.47 


0.73 


0.42 


0.68 


2 


Urban 


0.48 


0.16 


0.17 


0.79 


3.03 


<0.006 


11 


Suburban 


0.46 


0.36 


-0.25 


1.17 


1.27 


0.21 


2 


Mixed 


0.64 


0.28 


0.09 


1.18 


2.30 


0.03 


3 


Not specified 


0.56 


0.14 


0.28 


0.83 


3.95 


0.0005 


14 


Note: ES = ES from random-effect model, SE = SE of ES, t = 
statistical probability. 


■ t-statistic for the comparison, n 


= number of studies contributing to ES estimate, p = 


Table 3.H.5. Subanalysis for Spelling Outcomes by Population Density of Study Location 




Group 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








Urban 


0.58 


0.10 


0.38 


0.79 


5.63 


<0.001 


8 


Suburban 


0.71 


0.25 


0.21 


1.21 


2.79 


0.02 


1 


Mixed 


1.19 


0.29 


0.62 


1.75 


4.12 


0.002 


1 


Not specified 


0.46 


0.13 


0.20 


0.71 


3.50 


<0.006 


4 



Note: ES = ES from random-effect model, SE = SE of ES, t = t-statistic for the comparison, n = number of studies contributing to ES estimate, p = 
statistical probability. 
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Appendix 3.1: Coding for Articles Involving Code-Focused Interventions 



Study 


Age° 


Previous 

Literacy 

SkilP 


Intervention 

Type' 


Type 
of PA 
Training 
(Unit)'' 


Type 
of PA 
Training 
(Task)' 


Ball, E. W., & Blachman, B. A. (1988). Phoneme 
segmentation training: Effect on reading readiness. 
Annals of Dyslexia, 38, 208-225. 


1 


— 


2 


3 


1 


Beech, J. R., & Pedley, El. (1 994). Training letter-to- 
sound connections: The efficacy of tracing. Current 
Psychology, I3{2), 153-165. 


0 


2 


2 


— 


— 


Brady, S., Eowler, A., Stone, B., & Winbury, N. 
(1994). Training phonological awareness: A study 
with inner-city kindergarten children. Annals of 
Dyslexia, 44, 26-59. 


1 


2 


1 


3 


1 


Bus, A. G. (1986). Preparatory reading instruction 
in kindergarten: Some comparative research into 
methods of auditory and auditory-visual training of 
phonemic analysis and blending. Perceptual and 
Motor Skills, 62, 1 1-24. 


1 




2 


2 


3 


Bus (1986). 


1 


— 


5 


2 


3 


Byrne, B., & Eielding-Barnsley, R. (1991 ). Evaluation 
of a program to teach phonemic awareness to 
young children. Journal of Educational Psychology, 
83{A), 451-455. 


0 




1 


2 




Cary, L. SiVerhaeghe, A. (1994). Promoting 
phonemic analysis ability among kindergartners. 
Reading & Writing: An Interdisciplinary Journal, 6, 
251-278. 


0 


2 


1 


3 


3 


Cary & Verhaeghe (1 994). 


— 


— 


1 


— 


— 


Cary & Verhaeghe (1 994). 


— 


— 


1 


— 


— 


Castle, J. M., Riach, J., & Nicholson, T. (1994). 
Getting off to a better start in reading and spelling: 
The effects of phonemic awareness instruction 
within a whole language program. Journal of 
Educational Psychology, 86(3], 350-359. 


0 


1 


1 


1 


1 


Castle, Riach, & Nicholson (1994). 


— 


— 


2 


— 


— 


Castle, Riach, & Nicholson (1 994). 


— 


— 


2 


— 


— 
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ESs tor Primary Outcome Variables 



Language Spelling 



0.92 0.21 0.57 0.43 



0.56 



0.41 - 0.08 0.13 0.45 0.22 



0.16 



0.03 



1.65 



0.99 



1.81 



1.24 


3.58 


0.28 


2.00 



- 0.02 



- 0.24 - 0.17 



- 0.39 
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Study 


Age° 


Previous 

Literacy 

SkilP 


Intervention 

Type' 


Type 
of PA 
Training 
(Unit)'* 


Type 
of PA 
Training 
(Task)' 


Chera, R, & Wood, C. (2003). Animated multimedia 
"talking books" can promote phonological 
awareness in children beginning to read. Learning 
and Instruction, 13, 33-52. 


0 




5 


1 




Cunningham, A. (1990). Explicit versus implicit 
instruction in phonemic awareness. Journal of 
Experimental Child Psychology, 50, 429-444. 


1 


3 


1 


2 


3 


Diaz-Guerrero, R., & Eloltzman, W. El. (1974). 
Learning by televised "Plaza Sesame" in Mexico. 
Journal of Educational Psychology, 66{b], 632-643. 


0 


— 


3 


— 


— 


Eox, B., 8i Routh, D. K. (1984). Phonemic analysis 
and synthesis as word attack skills: Revisited. 
Journal of Educational Psychology, 7^6), 1059- 
1064. 


1 


2 


2 


2 


3 


Euchs, D., Euchs, L. S., Thompson, A., Al Otaiba, 
S., Yen, L, Yang, N. J., et al. (2001 ). Is reading 
important in reading readiness programs? A 
randomized field trial with teachers as program 
implementers. Journal of Educational Psychology, 
93{2], 251-267. 


1 




2 


1 


3 


Euchs, Euchs, Thompson, Al Otaiba, Yen, Yang, et al. 
(2001). 


1 


— 


5 


1 


3 


Euchs, D., Euchs, L. S., Thompson, A., Al Otaiba, 

S., Yen, L, Yang, N. J., et al. (2002). Exploring the 
importance of reading programs for kindergartners 
with disabilities in mainstream classrooms. 
Exceptional Children, 68(3], 295-31 1 . 


1 




2 


3 


2 


Euchs, D., Euchs, L. S., Thompson, A., Al Otaiba, S., 
Yen, L, Yang, N.J.,etal. (2002). 


1 


— 


5 


3 


2 


Goldstein, D. M. (1976). Cognitive-linguistic 
functioning and learning to read in preschoolers. 
Journal of Educational Psychology, 68(G], 680-688. 


0 


2 


5 


2 


2 


Eladdock, M. (1976). Effects of an auditory and an 
auditory-visual method of blending instruction on 
the ability of prereaders to decode synthetic words. 
Journal of Educational Psychology, 68(B], 825-831 . 


0 


2 


1 




3 


Eladdock (1976). 


0 


2 


2 


— 


3 
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ESs tor Primary Outcome Variables 



Language Spelling 

0.36 - 0.22 



1.92 



0.49 



0.83 



0.90 



1.22 



0.49 0.19 



1.50 


0.82 


0.58 


0.05 


- 0.50 


- 0.43 


0.40 


0.74 


0.11 


1.32 






1.01 







1.45 
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Study Age° 


Previous 

Literacy 

SkilP 


Intervention 

Type' 


Type 
of PA 
Training 
(Unit)'* 


Type 
of PA 
Training 
(Task)' 


Hecht, S. A., & Close, L. (2002). Emergent literacy 1 

skills and training time uniquely predict variability 
in responses to phonemic awareness training 
in disadvantaged kindergartners. Journal of 
Experimental Child Psychology, 82, 93-1 1 5. 


2 


5 


2 


3 


Kjeldsen, A.-C., Niemi, R, & Olofsson, A. (2003). 1 

Training phonological awareness in kindergarten 
level children: Consistency is more important than 
quantity. Learning and Instruction, 349-365. 




2 


3 


3 


Korkman, M., & Peltomaa, A. K. (1993). Preventive 1 

treatment of dyslexia by a preschool training 
program for children with language impairments. 

Journal of Clinical Child Psychology, 22(2], 277-287. 




2 






Lennon, J. E., & Slesinski, C. (1999). Early 1 

intervention in reading: Results of a screening and 
intervention program for kindergarten students. 

School Psychology Review, 28(3], 353-364. 




5 


3 


3 


Lonigan, C. J., Driscoll, K., Phillips, B. M., Cantor, 0 

B.G., Anthony., J. L, & Goldstein, El. (2003). 

A computer-assisted instruction phonological 
sensitivity program for preschool children at-risk 
for reading problems. Journal of Early Intervention, 

25(4), 248-262. 




1 


3 


3 


Lundberg, 1., Erost, J., & Peterson, 0. (1 988). 1 

Effects of an extensive program for stimulating 
phonological awareness in preschool children. 

Reading Research Quarterly, 23(3), 263-284. 




1 


3 


3 


Lyster, S. El. (2002). The effects of morphological 1 

versus phonological awareness training in 
kindergarten on reading development. Reading & 

Writing: An Interdisciplinary Journal, 15, 261-294. 


2 


2 


3 


3 


Martinussen, R. L., & Kirby, J. R. (1 998). Instruction 1 

in successive and phonological processing to 
improve the reading acquisition skills of at-risk 
kindergarten children. Developmental Disabilities 
Bulletin, 26(2), 19-39, 




2 


1 


3 
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ESs tor Primary Outcome Variables 



PA 


AK 


Oral 

Language 


Reading 


Spelling 


1.13 


0.14 


0.23 


1.09 


1.19 


1.60 


0.37 


- 0.10 


0.48 




0.40 




0.41 


0.78 




0.90 


0.89 




0.64 




0.41 


0.65 


- 0.13 


0.08 






0.32 




0.46 


0.45 


0.71 


0.37 


0.00 




0.20 


0.65 
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Study 


Age° 


Previous 

Literacy 

SkilP 


Intervention 

Type' 


Type 
of PA 
Training 
(Unit)'* 


Type 
of PA 
Training 
(Task)' 


Mioduser, D„ Tur-Kaspa, H„ & Leitner, 1. (2000). 

The learning value of computer-based instruction of 
early reading skills. Journal of Computer Assisted 
Learning, 16, 54-63. 


1 


1 


5 


1 




Murray, B. A. (1998). Gaining alphabetic insight: Is 
phoneme manipulation skill or identity knowledge 
causal? Journal of Educational Psychology, 90{3), 
461-475. 


1 




1 






O'Connor, R. E. (1999). Teachers learning ladders to 
literacy. Learning Disabilities Research and Practice, 
?4(4), 203-214. 


1 


— 


5 


1 


3 


O'Connor, R. E., Jenkins, J. R., Leicester, N., 

& Slocum, T. A. (1993). Teaching phonological 
awareness to young children with learning 
disabilities. Exceptional Children, 5^6), 532-546 


0 




1 


1 


3 


O'Connor, R. E., Notari-Syverson, A., & Vadasy, P. 

(1 996). Ladders to literacy: The effects of teacher- 
led phonological activities for kindergarten children 
with and without disabilities. Exceptional Children, 
fij(1), 117-130. 


1 


1 


5 


1 


3 


O'Connor, R. E., Notari-Syverson, A., & Vadasy, 

P. (1998). Eirst-grade effects of teacher-led 
phonological activities in kindergarten for children 
with mild disabilities: A follow-up study. Learning 
Disabilities Research and Practice, ?3(1 ), 43-52. 


1 




5 


3 


3 


Oudeans, M. K. (2003). Integration of letter-sound 
correspondences and phonological awareness 
skills of blending and segmenting: A pilot study 
examining the effects of instructional sequence 
on word reading for kindergarten children with 
low phonological awareness. Learning Disability 
Quarterly, 26, 258-280. 


1 


2 


2 


2 


2 


Peterson, M. E., & Elaines, L. P. (1992). Orthographic 
analogy training with kindergarten children: Effects 
on analogy use, phonemic segmentation, and letter- 
sound knowledge. Journal of Reading Behavior, 
24(1), 109-127. 


1 




5 


1 
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ESs tor Primary Outcome Variables 



PA 


AK 


Oral 

Language 


Reading 


Spelling 


1.19 


2.05 




1.90 




0.06 






0.17 




0.79 




1.42 


0.58 




0.72 


0.76 


0.33 


0.39 


0.99 




0.25 


0.17 


0.26 


0.80 


0.85 


0.32 






0.51 




1.21 


0.83 




0.74 
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Study Age° 


Previous 

Literacy 

SkilP 


Intervention 

Type' 


Type 
of PA 
Training 
(Unit)'* 


Type 
of PA 
Training 
(Task)' 


Roberts, T. A. (2003). Effects of alphabet-letter 0 

instruction on young children's word recognition. 

Journal of Educational Psychology, S5(1 ), 41-51 . 


1 


2 


1 


— 


Rohrlack, C. R., Bell, B. J., & McLaughlin, T. F. 1 

(1 982). The value of auditory skills for reading 
readiness programs. Educational Research 
Quarterly 7t1), 41-47. 




1 




2 


Saint-Laurent, L., & Giasson, J. (2001 ). Effects 1 

of a multicomponent literacy program and of 
supplemental phonological sessions on at- 
risk kindergartners. Educational Research and 
Evaluation, 7(1), 1-33. 


2 


1 


3 


3 


Saint-Laurent & Giasson (2001 ). 1 


2 


5 


3 


3 


Schneider, W., Ennemoser, M., Roth, E., & Kuspert, 1 

P. (1999). Kindergarten prevention of dyslexia: 

Does training in phonological awareness work for 
everybody? Journal of Learning Disabilities, 32{b], 
429-436. 


3 


1 


3 


3 


Schneider, W., Kuspert, R, Roth, E., Vise, M., & 1 

Marx, El. (1997). Short- and long-term effects of 
training phonological awareness in kindergarten: 

Evidence from two German studies. Journal of 
Experimental Child Psychology, BB, 311-340. 




1 






Schneider, Kuspert, Roth, Vise, & Marx (1997). 1 


— 


1 


— 


— 


Schneider, W., Roth, E., & Ennemoser, M. (2000). 1 

Training phonological skills and letter knowledge in 
children at risk for dyslexia: A comparison of three 
kindergarten intervention programs. Journal of 
Educational Psychology, 92,1], 284-295. 




1 


3 


1 


Schneider, Roth, & Ennemoser (2000). 1 


— 


2 


3 


1 


Schneider, Roth, & Ennemoser (2000). 1 


— 


3 


3 


1 


Silva, C., & Alves-Martins, M. (2002). Phonological 1 

skills and writing of presyllabic children. Reading 
Research Quarterly, 57(4), 466-483. 


2 


1 


1 


1 


Silva & Alves-Martins (2002). 1 


2 


5 


1 


1 
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ESs tor Primary Outcome Variables 



PA 


AK 


Lan°guSge 


Spelling 


0.55 


1.00 


- 0.41 




0.16 


0.25 






0.28 



0.34 






0.31 


1.04 




0.26 


0.53 


0.65 


0.12 


0.21 





0.70 


0.05 


0.08 


0.05 


0.77 






- 0.61 



CD 

1 


CO 

CD 

1 


0.48 


- 0.52 


1.62 




1.05 
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Study 


Age° 


Previous 

Literacy 

SkilP 


Intervention 

Type' 


Type 
of PA 
Training 
(Unit)'* 


Type 
of PA 
Training 
(Task)' 


Silva, C., & Alves Martins, M. (2003). Relations 
between children's invented spelling and the 
development of phonological awareness. 
Educational Psychology, 23{] ), 3-1 6. 


1 


2 


5 


1 




Solity, J. (1 996). Phonological awareness: Learning 
disabilities revisited? Educational and Child 
Psychology, 13{3], 103-113. 


0 


— 


1 


3 


3 


Solity (1996). 


0 


— 


2 


3 


3 


Solilty, J., Deavers, R., Kerfoot, S., Crane, G., & 
Cannon, K. (1999). Raising literacy attainments 
in the early years: The impact of instructional 
psychology. Educational Psychology, ?^4), 373-397. 


0 




5 




3 


Tangel, D. M., & Blachman, B. A. (1992). Effect of 
phoneme awareness instruction on kindergarten 
children's invented spelling. Journal of Reading 
Behavior, 24{2], 233-261 . 


1 




2 


3 




Torgesen, J., & Davis, C. (1996). Individual 
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ESs tor Primary Outcome Variables 



Language Spelling 

1.36 



0.43 




0.50 




0.68 




0.79 




0.47 


0.86 


0.62 


0.42 


1.76 


0.49 


0.57 


0.93 



0.86 



1.91 0.18 



- 1.57 



0.16 



0.50 0.25 



- 0.14 



0.69 
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1-1 5.f 




1 







" Age of children in intervention (0 = preschool, 1 = kindergarten). 

^ Literacy ievei of children in intervention ( 1 - iittle or no AK and not reader, 2 = moderate to high AK and not reader, 3 = reader). 

‘ Type of intervention (1 = PA training only, 2 = PA training and letter-knowledge training, 3 = ietter-knowledge training oniy 4 = PA training and 
phonics training). 

Linguistic unit of analysis used in PA training activity ( 1 = subphoneme, 2 = phoneme, 3 = both subphoneme and phoneme). 

‘ Operation or manipulation used in PA training ( 1 = anaiysis, 2 = synthesis, 3 = both analysis and synthesis). 

' This study is combined with the originai study: Whitehurst, G. J., Epstein, J. N., Angell, A. C., Payne, A. C., Crone, D. A. & Fischel, J. £ (1994). 
Outcomes of an emergent literacy intervention in Head Start. Journal of Educational Psychology, 8B, 542-555. 
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ESs tor Primary Outcome Variables 



Language Spelling 



1.04 







0.42 




0.69 


- 0.63 


0.52 


1.11 


0.59 


0.50 


0.97 




0.75 









0.09 



DEVELOPING EARLY LITERACY: REPORT OF THE NATIONAL EARLY LITERACY PANEL 



151 



Chapter 4 
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Shared-reading practices — a parent reading a picture book with a toddler or a teacher reading a 
book to a class of preschoolers — are reading practices that are widely recommended to promote 
language and other skills related to early literacy development. Shared-reading activities are 
often recommended as the single most important thing adults can do to promote the emergent 
literacy skills of young children. Scarborough and Dobrich (1994; see also Bus, van Ijzendoorn, 
& Pellegrini, 1995) provided a summary of studies that examined the effect of shared reading on 
young children’s emergent literacy skills, and their results called into question the positive effects 
often claimed for reading or sharing picture books with young children. 

Accordingly, the National Early Literacy Panel (NELP) examined the effects of interventions 
that primarily or entirely focused on shared reading. These shared-reading interventions included 
those that involved parents, teachers, or the combination of parents and teachers implementing 
some form of shared reading with children individually or in groups. To be included in 
NELP’s analysis, a study that examined the effects of shared reading had to use a group design 
(randomized control trial [RCT] or quasiexperimental design [QED]) that was not seriously 
confounded, and it had to evaluate the effectiveness of the intervention with outcome measures 
of conventional literacy skills (i.e., decoding, reading comprehension, or spelling) or skills that 
NELP identified as predictors of later conventional literacy skills. The studies also had to have 
appeared in a refereed journal; they had to include sufficient information to allow an effect size 
(ES) to be calculated; and those studies that used QEDs had to provide evidence of the group’s 
initial comparability. Nineteen studies on the impacts of shared-reading interventions met 
these criteria. 
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The studies included in NELP’s analysis of shared-reading interventions differ from those 
included in the earlier Scarborough and Dobrich (1994) and Bus et al. (1995) reports in a 
number of ways. NELP s analysis considered only those studies that had undergone some 
independent scientific review, included studies of both preschool and kindergarten children, and 
included only studies that evaluated the effects of interventions. NELP subjected the studies to 
a more rigorous set of screening criteria to increase the likelihood that the effects were causally 
interpretable, and finally, NELP included studies that had not yet been published at the time of 
the earlier review. 

Children, in most of these studies, were exposed to some kind of a short-term (i.e., one to six 
months) shared-reading intervention that either represented a substantial increase in frequency 
of shared-reading activities or a change in the style of shared-reading activities (such as engaging 
the children actively in telling the story rather than being passive listeners). There were many 
variations on these procedures, with some delivered by teachers and others by parents. Some 
studies examined whole-class interventions; one study examined the impact of providing books 
and information to parents during well-baby pediatrician visits; and two other studies examined 
the impact of computerized storybook interventions. Children in the comparison groups in these 
studies usually received less exposure to shared reading than did the children in the experimental 
group, and the shared reading they did receive rarely involved more than the adult just reading 
books to children. In most cases, the researcher did not specify or control what the children 
experienced in the comparison-group condition, meaning that these childrens exposures to 
shared reading were to the usual practices of their teachers or parents. Consequently, these studies 
provide comparisons of some kind of intensified or improved effort to read to children with the 
usual kinds of shared reading that children commonly experience. 



Overall Estimates of Intervention Impacts 

Listed in Table 4.1 are the overall impacts of the shared-book interventions across all of the 
outcome variables that NELP considered. This table includes, for each outcome variable 
(presented in alphabetical order), the number of studies that contributed to the ES estimates, an 
estimate of the ES based on a fixed-effect model, an estimate of the ES based on a random-effect 
model, ^ the 95 percent upper and lower bounds of the ES based on the random-effect model, 
and the statistical significance of the ES from the random-effect model. Most of the shared- 
reading intervention studies measured the impact of the interventions on oral language skills (16 
studies). Fewer studies examined the impact of these interventions on phonological awareness 
(PA) (two studies), general cognitive ability (one study), alphabet knowledge (AK) (two studies), 
print knowledge (four studies), reading readiness (one study), or writing (one study). No studies 
in this category of interventions examined the impact of the intervention on memory; rapid 
automatic naming (RAN), reading, spelling, or visual processing as an outcome variable. 

As can be seen in Table 4.1, shared-reading interventions had moderate effects on measures of 
oral language and print knowledge. The one study with writing as an outcome measure also 
reported a moderate impact. No other effect was statistically significant or of a size that would 

' Except where noted, all ES estimates and associated statistics are based on a random-effect model. 
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be considered substandvely important. These studies indicate that shared-reading interventions 
can have a significant, substantial, and positive impact both on young childrens oral language 
skills and on young children’s print knowledge. Shared-reading interventions appear to have no 
impact on young children’s PA skills or their AK; however, there have been too few studies using 
these — or other — outcome measures to provide a reliable estimated ES. 



Table 4. 1. Estimates of Effect Sizes Across Outcome Domains for Interventions Involving Shared Reading or 
Sharing Books with Young Children for Each Dependent Variable 



Dependent Variable 


Fixed ES 


Random ES 




95% Cl 


Nof 

Studies 


pfor ES 








Lower 

Bound 


Upper 

Bound 






AK 


-0.06 


-0.06 


-0.47 


0.35 


2 


0.78 


Cognitive ability 


0.10 


0.10 


-0.21 


0.41 


1 


0.52 


Oral language 


0.66 


0.73 


0.27 


1.20 


16 


0.002 


PA 


0.11 


0.11 


-0.15 


0.35 


2 


0.42 


Print knowledge 


0.51 


0.50 


0.28 


0.73 


4 


0.0001 


Readiness 


-0.14 


-0.14 


-0.64 


0.36 


1 


0.58 


Writing 


0.52 


0.52 


0.23 


0.81 


1 


0.0005 



Note: Cl = Cl for random-effect model. 



The largest impact of shared reading was on oral language outcomes, with an average ES of 
0.73. This result means that, on average, children who received a shared-reading intervention 
scored, on oral language, more than 0.7 of a standard deviation higher than children who had 
not received such instruction. To put this in context, if the average children who were not read to 
in the enhanced format scored 1 00 on a standardized test of oral language (with a mean of 1 00 
and a standard deviation of 15), then the average children who were read to in these enhanced 
or extended ways would score 1 1 1 on the test (i.e., the difference between scoring at the 77th 
percentile versus scoring at the 50th percentile). 

One study in this set, an examination of the Reach Out and Read (ROR) program, had a large, 
and perhaps misleading, ES (High, LaGasse, Becker, Ahlgren, & Gardner, 2000). This study 
was a quasiexperiment that yielded a statistically significant and larger ES than the average of 
all randomized studies combined. In this study, the researchers did not directly assess student 
language development but asked parents to estimate their children’s vocabulary performance. 
When one study has such a different result, it is common practice to conduct the analysis again 
with that outlier excluded. With that study removed from the oral language analysis, the ES 
estimate shrinks to 0.57, making it smaller but still showing that sharing books with young 
children has a significant, moderate impact on children’s learning. See Table 4.2 for the results of 
this analysis with the outlier effect removed. 
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Table 4.2. Effect Sizes for Oral Language Outcomes for Shared-Reading Interventions 



Dependent Variable 


Fixed ES 


Random ES 




95% Cl 


Nof 

Studies 


pfor ES 








Lower 

Bound 


Upper 

Bound 






Oral language with outlier 
study included 


0.66 


0.73 


0.33 


0.60 


16 


0.002 


Oral language with outlier 
study excluded 


0.47 


0.57 


-0.23 


0.90 


15 


0.001 



Note: Cl = Cl for random-effect model. 



Estimates of Intervention Impacts by Characteristics of Interventions 
and Populations 

What kinds of shared-reading experiences have made this difference? In addition to the analyses 
on the overall effects of shared-reading interventions on children’s early literacy and language 
skills, analyses were conducted to determine what effects variations in implementations had 
or what kinds of children benefited most from this kind of support. These analyses could be 
conducted only for outcome variables for which there was a sufficient number of studies to allow 
estimation of effects by subcategory. For this reason, only studies that measured the impact of 
shared reading on oral language development could be analyzed. But even with oral language, 
there were only 16 studies. Consequently, it is likely that not all of the subanalyses reported here 
reflect independent analyses of separate variables; that is, in some cases, the subdivisions of the 
various studies overlap with other subdivisions of the studies. 

The first subanalysis was conducted to determine whether there were any differences in the 
ESs based on the type of study design (RCT versus QED). The results of this comparison 
are shown in Table 4.3. All but one of the studies that included oral language as an outcome 
variable randomly assigned children to the treatment or control group. The ES estimate for the 
quasiexperiment was larger than for the other studies (Q[l, 13] = 16.73, p < 0.0001), suggesting 
that this study was a relative outlier in terms of the results obtained. Part of the impact identified 
in that study may have been due to preexisting but unidentified differences between the 
experimental and control groups. 



Table 4.3. Effect Sizes for Oral Language Outcomes for Study Classification Based on Type of Study Oesign 



Study Design 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








RCT 


0.56 


0.15 


0.26 


0.86 


3.65 


15 


0.003 


OED 


2.87 


0.54 


1.80 


3.93 


5.29 


1 


0.0002 



NELP’s analysis of predictors of conventional forms of reading and writing growth revealed 
that some oral language skills were better predictors of later reading outcomes than were others. 
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Specifically, measures of more complex oral language skills (such as grammar, the ability to define 
vocabulary, and listening comprehension) and composite measures that included these skills 
were stronger predictors of later reading achievement than were measures of simple vocabulary. 
For this reason, the panel considered whether there was a different impact of shared-reading 
interventions on composite measures of oral language than on measures of simple vocabulary. 

The results of this analysis, with the one outlier study excluded, are shown in Table 4.4. Although 
the ES estimate for measures of simple vocabulary was almost twice that for composite measures 
of oral language, this difference was not statistically reliable, meaning that it cannot be said 
with certainty that there was a difference. With the ROR study included, the average ES for 
vocabulary was 0.84, but the differences between effects on vocabulary and composite measures 
of oral language still were not statistically significant. 



Table 4.4. Effect Sizes of Shared-Reading Interventions on Measures of Simple Vocabulary and Composite 
Measures of Oral Language, Excluding Study on Reach Out and Read 



Outcome Measure 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Vocabulary 


0.60 


0.23 


0.16 


1.05 


2.65 


9 


0.008 


Composite oral language 


0.35 


0.28 


-0.20 


0.89 


1.26 


5 


0.21 



Analysis of Intervention Effects by Age and Risk Status of Children 

Studies involving the impacts of shared-reading interventions included a broad age range of 
children, ranging from studies involving two-year-olds to studies involving kindergarten-age 
children. The impact of shared-reading interventions may differ for younger and older children 
for developmental or experiential reasons. To examine whether shared-reading interventions 
had different effects on children at different age levels, where possible, studies were divided into 
two groups. One group of studies focused on children who were not yet in kindergarten, and 
the other studies focused on children who were in kindergarten. Studies that combined children 
from the two age groups were excluded from these analyses. Estimates of the impact of shared- 
reading interventions for preschool- age or younger children and kindergarten-age children are 
shown in Table 4.5. There was no statistically significant difference in the ES estimates for older 
versus younger children on oral language outcomes (Q[l, 13] = 0.10, = 0.86). Shared-reading 

interventions were equally effective with children across these age ranges. 



Table 4.5. Effect Sizes for Oral Language Outcomes for Study Classification Based on Ages of Children in 
Sample 



Child Age Group 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Prekindergarten 


0.75 


0.24 


0.29 


1.22 


3.15 


13 


0.0008 


Kindergarten 


0.66 


0.46 


-0.25 


1.57 


1.43 


3 


0.18 
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To examine whether shared-reading interventions had smaller or larger effects for children based 
on their risk status, studies were divided on the basis of whether the majority of the children in a 
study were at risk. Classification of studies were done either using the study authors’ classification 
of the study sample or based on the description of the study sample (e.g., children attending 
Head Start were classified as at risk because they are growing up in poverty). Studies with 
combined samples across this dimension (i.e., children both at risk and not at risk included in 
the study sample) were not included in this analysis. Estimates of the impact of shared-reading 
interventions for children at risk and those not at risk are shown in Table 4.6. There appeared 
to be a larger effect for studies that focused on children who were not at risk, but the difference 
in the ES estimates was not statistically reliable (Q[l, 13] = 0.56, p = 0.45), and the confidence 
intervals (CIs) overlapped substantially. 



Table 4.6. Effect Sizes for Oral Language Outcomes for Study Classification Based on Risk Status of 
Children in Sample 



Child Risk Status 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








At risk 


0.47 


0.40 


-0.31 


1.26 


1.19 


12 


0.26 


Not at risk 


0.82 


0.25 


0.35 


1.31 


3.37 


4 


0.005 



Analysis of Intervention Effects by Characteristics of Shared-Reading 
Intervention 

The shared-reading interventions that were considered in these studies differed in the degree 
to which they were interactive. One common form of interactive shared reading is known as 
dialogic reading (DR). In DR, the adult reader asks the child or children questions about the 
story or the pictures in the book and provides feedback to the child or children in the form of 
repetitions, expansions, and modeling of answers. In DR, the adult tries to facilitate the child’s 
active role in telling the story rather than foster passive listening. Studies were separated based on 
whether the reading was interactive like DR. Results for this analysis are shown in Table 4.7. 



Table 4.7. Effect Sizes for Oral Language Outcomes for Study Classification Based on Type of Shared 
Reading Used in Intervention 



Type of Reading 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Dialogic 


0.60 


0.27 


0.06 


1.14 


2.18 


9 


0.05 


Not dialogic 


0.78 


0.31 


0.18 


1.38 


2.56 


7 


0.02 



As noted in Table 4.7, the ES estimates for both DR interventions and non-DR-style 
interventions were moderately sized and statistically significant. The ES estimate for non-DR 
interventions was slightly higher than that for DR interventions; however, this difference was not 
statistically reliable (Q[l, 14] = 0.20, p = 0.65). The noninteractive shared-reading interventions. 
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lacking the guidance and support of the more thoroughly structured DR interventions, were 
effective; however, this was due, in part, to a large ES for the outlier study (High et al., 2000). 
Consequently, the analysis reported in Table 4.7 was conducted again, this time with the ROR 
study excluded. The results of this analysis are shown in Table 4.8. The removal of the ROR 
study decreased the ES estimate for non-DR studies from 0.78 to 0.42 and rendered this finding 
nonsignificant (meaning that the groups in the noninteractive approaches did not actually do 
statistically better than the control groups when the ROR study was removed from this analysis). 
Even with this adjustment, however, the difference in ES estimates for studies using DR and 
those not using DR was not statistically reliable (Q[l, 13] = 0.36, p = 0.55). 



Table 4.8. Effect Sizes for Oral Language Outcomes for Study Classification Based on Type of Shared 
Reading Used in Intervention, Excluding Study on Reach Out and Read 



Type of Reading 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Dialogic 


0.59 


0.20 


0.20 


0.98 


2.99 


9 


0.01 


Not dialogic 


0.41 


0.24 


-0.06 


0.87 


1.72 


6 


0.11 



Children’s early childhood education teachers, children’s parents, and combinations of teachers 
and parents have conducted shared-reading interventions. Table 4.9 lists the ES estimates from 
interventions in which teachers, parents, or both teachers and parents provided the shared- 
reading intervention (or the computerized intervention was used). There was no statistically 
reliable difference in ESs depending on how the shared reading was delivered (Q[3, 12] = 

0.16, p = 0.16). Comparison of the studies involving parents reading to their children and 
studies involving both parents and teachers doing the reading did not have statistically reliable 
differences in ESs (the CIs overlap). When the ROR study (involving parent reading) was 
excluded from the analysis, the estimated ES for parent-provided reading was reduced to 0.57 
(;-=0.l6). 



Table 4.9. Effect Sizes for Oral Language Outcomes for Study Classification Based on Agent of Intervention 



Agent of Intervention 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Parent 


1.35 


0.40 


0.56 


2.14 


3.36 


3 


0.006 


Teacher 


0.84 


0.32 


0.21 


2.60 


2.60 


5 


0.023 


Parent and teacher 


0.29 


0.30 


-0.29 


0.88 


0.99 


6 


0.34 


Computer 


0.36 


0.50 


0.61 


1.34 


0.73 


2 


0.48 



Other aspects of the shared-reading interventions were coded to identify possible moderators 
of the effectiveness of shared-reading interventions. In some studies, books were provided as 
a part of the intervention to support the shared-reading effort. Estimates for ESs depending 
on whether books were provided are shown in Table 4.10. If books were provided as a part of 
the intervention, the estimated ES was 0.78, but this dropped to 0.50 with the ROR study 
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eliminated. If books were not provided as a part of the intervention, the estimated ES was 0.55, 
but the difference between ESs with (Q[l, 14] = 0.31, = 0.58) or without (Q[l, 13] = 0.02, p = 

0.88) the ROR study was not statistically reliable. 



Table 4. 10. Effect Sizes for Oral Language Outcomes for Study Classification Based on Whether Books 
Were Provided as a Part of the Intervention 



Books Provided 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








No 


0.55 


0.31 


-0.05 


1.15 


1.80 


1 


0.09 


Yes 


0.78 


0.27 


0.25 


1.31 


2.88 


9 


0.01 



Only five studies reported the amount of training provided to parents or teachers for the shared- 
reading intervention. The amount of training reported in these studies ranged from 50 to 180 
minutes. Analyses of these five studies revealed that there was no significant relationship between 
amounts of training provided and the ES estimate for those studies ((3 = 0.61, p - 0.32). Nine 
studies reported the approximate total amount of time that children were exposed to the shared- 
reading interventions. Reports of amount of reading in these studies ranged from 1 12 to 1,500 
minutes. There was no relationship between the amount of reading and variation in ES ((3 = 
-0.01, = 0.98). 

Analysis of Intervention Effects by Demographics of Study Samples 

NELP considered whether there were different impacts for shared-reading efforts with children 
from different demographic groups, including socioeconomic status (SES), ethnicity, and the 
population density of the location where the study was conducted (i.e., rural, urban, suburban, 
mixed, unknown). In the majority of studies, either these demographic characteristics were not 
reported or the studies examined samples of children with mixed demographic characteristics. 

Results of the subanalyses of the impact of SES level in shared-reading interventions are shown 
in Table 4.1 1. ES estimates for oral language outcomes did not vary significantly as a result of 
children’s economic status (Q[2, 9] = 0.23, p = 0.89). Most studies that could be coded for SES 
included children from low-SES backgrounds, and there were too few studies with children from 
other economic levels to generate reliable ES estimates. Notably, the 9 studies involving only 
children from low-SES backgrounds produced a large and statistically significant ES estimate 
of 0.79. 

Results of the ethnicity subanalysis are shown in Table 4.12. There was no statistically reliable 
difference in ESs associated with ethnicity (Q[3, 1 1] = 0.28, p = 0.96). However, there were 
only single instances of shared-reading interventions aimed at samples that were majority 
Caucasian or that were majority Hispanic or Latino. Consequently, there were too few studies 
with these ethnic classifications from which to generate reliable ES estimates. Three studies 
of shared-reading interventions were conducted with children who were majority African 
American. The remaining 1 0 studies were conducted with children with diverse or unspecified 
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ethnic classification. CIs for studies involving either African American or mixed or unknown 
populations overlapped substantially. Consequently, there was no evidence that ethnicity 
moderated the impacts of shared-reading interventions. 



Table 4. 1 1. Effect Sizes for Oral Language Outcome for Shared-Reading Study Classification Based on 
Socioeconomic Status of Study Sample 



SES 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Low 


0.79 


0.27 


0.25 


1.32 


2.89 


9 


0.02 


Not low 


0.85 


0.59 


-0.31 


2.01 


1.44 


2 


0.18 


Mixed 


0.41 


0.79 


-1.12 


1.95 


0.52 


1 


0.61 



Table 4. 12. Effect Sizes for Oral Language Outcome for Shared-Reading Study Classification Based on 
Ethnicity of Study Sample 



Ethnicity 


Mean ES SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Caucasian 


0.41 


0.77 


-1.13 


1.95 


0.52 


1 


0.61 


African American 


0.70 


0.47 


-0.22 


1.63 


1.49 


3 


0.17 


Hispanic or Latino 


1.02 


0.90 


-0.74 


2.79 


1.14 


1 


0.28 


Mixed or unknown 


0.75 


0.26 


0.25 


1.26 


2.93 


10 


0.014 


Results of the analysis 


based on 


population 


density are 


shown in Table 4.13. There was 


no 



statistically reliable difference in ESs associated with population density (Q[3, 1 1] = 3.67, p = 
0.30). The samples from the majority of studies were coded as unknown or urban population 
densities. There were too few instances of rural or suburban population density to generate 
reliable ES estimates. However, the CIs for the studies classified as urban or unknown 
overlapped. Therefore, there was no evidence that population density moderated the impact of 
shared-reading interventions. 



Table 4. 13. Effect Sizes for Oral Language Outcome for Shared-Reading Study Classification Based on 
Population Oensity of Study Sample 



Population Density 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Rural 


-0.12 


0.69 


-1.47 


1.24 


-0.17 


1 


0.87 


Urban 


1.05 


0.28 


0.51 


1.59 


3.83 


7 


0.003 


Mixed 


0.96 


0.78 


-0.56 


2.49 


1.24 


1 


0.24 


Unknown 


0.47 


0.31 


-0.13 


1.10 


1.55 


6 


0.15 
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Summary and Conclusions 

Results from this meta- analysis of the impacts of shared-reading interventions on the early 
literacy skills of young children indicated that these interventions yielded moderate effects on 
oral language skills and print knowledge. For oral language skills, these effects were robust across 
variations in the type of shared-reading intervention and the childrens ages or their risk status. 
Although it is possible that shared reading could affect other aspects of children’s literacy and 
language development, only four studies even included print knowledge as an outcome variable, 
and even fewer studies included any other variable. Therefore, it was not possible to determine 
whether there were other benefits of shared reading. Also, at this time, there are few studies 
that allow fine-grained analysis of the impact of population variables, such as SES, ethnicity, or 
population density. Given the lack of evidence that sample characteristics moderate the effects 
of shared-reading interventions, along with the inclusion of diverse samples in the existing 
studies and the consistency of results across studies, it seems reasonable at this time to make 
such interventions available to a broad range of children. 

Given the ubiquity of both the practice of and the recommendation for shared reading in early 
childhood education settings, it is somewhat surprising that more studies have not investigated 
the impact of these practices. Although it is clear that shared reading improves oral language skills 
and print knowledge, there is not yet evidence that shared reading promotes the development 
of other emergent literacy skills, and there is no evidence that shared reading promotes any 
improvement in conventional literacy skills. Although it is often claimed that reading to children 
improves their reading ability, too few studies have been conducted with emergent literacy 
outcome measures (such as PA, AK, readiness, and writing) or conventional literacy outcome 
measures (such as decoding, reading comprehension, or spelling) to provide statistically reliable 
evidence that shared reading improves such skills (and, if so, which ones). Given these important 
gaps in what is known about the effectiveness of shared reading, it seems prudent to conclude 
that shared reading alone would not be a sufficient response to the literacy learning needs of 
young children. This would be particularly true for those at risk or who show weaknesses in those 
specific emergent literacy skills that have not been shown to improve due to reading to children 
(such as PA or AK). 

Based on available studies, it appears that shared-reading interventions are equally effective for 
children who are at risk of later academic difficulties and for children who are not at risk. Also, 
shared-reading interventions appear to be equally effective for older and younger children. 

Perhaps direct study of these variations would permit a more definitive answer concerning their 
influence. Given the existing pattern of results, it seems reasonable to conclude that shared 
reading is appropriate and useful for a very diverse group of young children. 

It is important to note, however, that, because of the limited number of studies available, our 
understanding of the impact of age, risk status, and agent of intervention is inadequate. This 
is partly due to the lack of studies reporting data in ways that allow these comparisons to be 
made and partly because the various moderators are confounded in the existing studies. For 
instance, most DR studies with younger children used parents as the agents of intervention and 
were conducted with middle- to upper-income families (e.g., Arnold et ah, 1994; Huebner, 

2000; Whitehurst et al., 1988). Most DR studies with older children were conducted using 
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teachers as the agents of intervention (i.e., in preschool or child-care settings) with children 
from low-income families (e.g., Lonigan & Whitehurst, 1998; Whitehurst, Arnold, et al., 1994; 
Whitehurst, Epstein, et al., 1994). Therefore, for meta- analysis, no one of these moderators can 
be separated from the other moderators. Likewise, analyses of duration of the interventions are 
confounded with age, agents of intervention, and risk status of the children. Therefore, it is not 
possible to examine combinations of these potential moderators with currently existing studies 
(e.g., the impact of parents using a shared-reading intervention for younger, at-risk children). 

Despite any analytical limitations, these studies indicate that shared-reading interventions 
provide early childhood educators and parents with a useful method for successfully stimulating 
the development of young children’s oral language skills. For some reason, the impact of shared- 
reading interventions is larger for vocabulary outcomes than for more complex aspects of oral 
language (such as grammar, narrative understanding, or listening comprehension) or broader 
measures of oral language that include aspects of both vocabulary and more complex oral 
language skills. Whether this is due to real differences in outcomes or to the nature of the shared- 
reading interventions that have been studied and the outcome measures used so far is as yet 
unknown. Additional research will be needed to better explain this finding. 

Future research needs to examine the types of shared-reading interventions that have been 
studied and how these interventions have been delivered. Interventions that used an interactive 
style of shared reading, such as DR, produced larger effects on children’s oral language outcomes 
than did noninteractive interventions, but these differences did not reach statistical significance. 
However, only studies using DR resulted in an average ES that was statistically significant. Direct 
studies of the contrast between interactive shared reading and noninteractive shared reading (e.g., 
see Lonigan et al., 1999) could help to clarify the meaning of this difference. For the existing 
studies, there were no significant differences in outcomes due to who delivered the shared-reading 
interventions, whether books were provided as part of the intervention, or how much the adults 
read to the children. It is important to note that statistical significance is not the only issue of 
importance in the context of a meta- analysis. Statistical significance — that is, the determination 
that an effect is sizable enough that it would unlikely have occurred by chance or normal 
variation — is affected by both the size of a difference and the number of observations (in the case 
of meta- analysis, the number of studies). The sizes of the differences found here for DR, agent 
delivering the intervention, amount of reading, and book availability were large enough to be of 
educational importance but were simply not found across a sufficiently large sample of studies to 
achieve statistical significance. 

For studies conducted in preschool or kindergarten classes, the teacher or other adult most often 
read to children in small groups. Notably, the estimated ESs for shared reading do not reflect 
the impact of the typical program of shared reading conducted in early childhood settings (e.g., 
whole-group shared reading during circle time), which was typically the comparison condition 
in studies of shared reading in schools. Consequently, the results of this analysis do not provide 
evidence that typical early childhood education classroom practices promote the development of 
oral language and print knowledge skills. 

Overall, the evidence supports the positive impact of shared-reading interventions that are more 
intensive in frequency and interactive in style on the oral language and print knowledge skills of 
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young children. At present, the number of studies in the literature that have examined specific 
groups of children (such as children from different SES backgrounds, different ethnicities, home 
languages, or living circumstances — i.e., rural versus urban) is not sufficient to allow an adequate 
analysis of how shared-reading interventions may result in larger or smaller effects on these 
groups. It seems reasonable to proceed with the idea that shared reading would help all or most 
subgroups of children, given the inclusion in these studies of mixed samples of children from 
different socioeconomic backgrounds, different ethnicities, and different living circumstances. 
Although the early childhood education field is interested in specific questions about successful 
interventions for children of low-income families, children from traditionally underrepresented 
ethnic groups, children who are English-language learners, or children growing up in rural or 
urban environments, studies focusing on shared reading with these groups have not yet been 
reported in sufficient frequency to allow definitive answers to these questions. Nevertheless, the 
existing studies provide no reason to expect substantially different patterns of results for these 
variables in future research. 
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Research Studies Synthesized in the Analysis of Shared-Reading Interventions 
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Appendix 4.A. Primary Coding of Studies with Oral Language 
Outcome Measures 



Study 


Type of 
Study’ 


Age'’ 


DR or Not 
DR’ 


Risk 

Status'* 


Ahlgren, 1., Becker, S., Gardner, A., High, P. C., & LaGasse, L. (2000). 
Literacy promotion in primary care pediatrics: Can we make a 
difference? Pediatrics, /05(4), 927-934. 


2 


1 


0 


1 


Arnoid, D. H., Epstein, J. N., Lonigan, C. J., & Whitehurst, G. J. 
(1994). Accelerating ianguage deveiopment through picture book 
reading: Repiication and extension to a videotape training format. 
Journai of Educationai Psychology, 8B{2], 235-243. 


1 


1 


1 


0 


Crain-Thoreson, C., & Daie, P. S. (1999). Enhancing iinguistic 111 1 

performance: Parents and teachers as book reading partners for 
chiidren with ianguage deiays. Topics in Early Childhood Special 
Education, ?fl(1 ), 28-40. 


Huebner, C. E. (2000). Promoting toddiers' ianguage deveiopment 
through community-based intervention. Journal of Applied 
Developmental Psychology, 2/(5), 513-535. 


1 


1 


1 


0 


Johnston, C. B. (1997). interactive storybook software: Effects 
on verbai deveiopment in kindergarten chiidren. Early Child 
Development and Care, /52 33-44. 


1 


2 


0 


1 


Kerr, B. M., Mason, J. M., Norris, S. P, & Phiiiips, L. M. (1 990). 
Effect of eariy iiteracy intervention on kindergarten achievement. 
National Reading Conference Yearbook, 39, 199-207. 


1 


2 


0 


0 


Lonigan, C. J., Anthony, J. L., Bioomfieid, B. G., Dyer, S. M., & 
Samwei, C. S. (1999). Effects of two shared-reading interventions 
on emergent iiteracy skiiis of at-risk preschooiers. Journal of Early 
Intervention, 22(4), 306-322. 


1 


1 


0 


1 


Lonigan, Anthony, Bioomfieid, Dyer, & Samwei (1999). Ill 1 


Lonigan, C. J. & Whitehurst, G. J. (1998). Relative efficacy of 
parent and teacher involvement in a shared-reading intervention 
for preschooi chiidren form iow-income backgrounds. Early 
Childhood Research Quarterly, 13{2], 263-290. 


1 


1 


0 


1 


Morrow, L. M., O'Connor, E. M., & Smith, J. K. (1990). Effects of 
a story reading program on the iiteracy deveiopment of at-risk 
kindergarten chiidren. Journal of Reading Rehavior, 22(3), 255-275. 


1 


2 


0 


1 


Taiiey, S., Lancey, D. E, & Lee, T. R. (1997). Chiidren, storybooks. 


1 


1 


0 


1 



and computers. Reading Horizons, 3^2), 117-128. 
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Oral 

Language^ 


SES' 


Ethnicity^ 


Population 

Density'' 


Mean ES 


1 


1 


4 


2 


2.87 


1 


2 


4 


5 


0.75 


1 




4 


5 


0.29 


1 


3 


1 


2 


0.41 


— 


1 


4 


5 


-.49 


2 


— 


4 


1 


CSI 

CD 

1 


1 


1 


2 


2 


- 0.14 


2 


1 


2 


2 


- 0.03 


1 


1 


2 


2 


0.43 


2 


— 


4 


2 


1.70 


2 


1 


4 


5 


0.24 
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Study 


Type of . ^ DR or Not 

Study" DR" 


Risk 

Status'* 


Valdez-Menchaca, M. C., & Whitehurst, G. J. (1992). Accelerating 
language development through picture book reading: A systematic 
extension to Mexican day care. Developmental Psychology, 28{6], 
1106-1114. 


1 1 1 


1 


Wasik, B. A., & Bond, M. A. (2001 ). Beyond the pages of a book: 
Interactive book reading and language development in preschool 
classrooms. Journal of Educational Psychology, 93(2], 243-250. 


1 1 1 


1 


Whitehurst, G. J., Arnold, D. S., Epstein, J. N., Angell, A. L, Smith, 
M., & Fischel, J. E. (1994). A picture book reading intervention 
in day care and home for children from low-income families. 
Developmental Psychology, 30(b], 679-689. 


1 1 1 


1 


Whitehurst, G. J., Ealco, E. L., Lonigan, C. J., Eischel, J. E., 
DeBaryshe, B. D., Valdez-Menchaca, M.C., et al. (1 988). 
Accelerating language development through picture book reading. 
Developmental Psychology, 24(4), 552-559. 


1 1 1 


0 


Whitehurst, G. J., Zevenbergen, A. A., & Zevenbergen, J. A. 
(2003). Effects of a shared-reading intervention on the inclusion 
of evaluative devices in narratives of children from low-income 
families. Applied Developmental Psychology, 24(1), 1-15. 


1 1 1 


1 



‘RCT= 1,QED = 2. 

‘ Preschool or younger = 1, kindergarten = 2. 

‘DFt= 1, NotDH = 0. 

‘'At risk = 1, notatrisk = 0. 

‘ Vocabulary = 1, composite measure = 2. 

' Low SES = 1, Not low SES = 2. 

^ Ethnicityll = Caucasian, 2 = African American, 3 = Hispanic or Latino, 4 = mixed or unknown). 
* Population density ( I = rural, 2 = urban, 3 = suburban, 4 = mixed, 5 = unknown). 
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SES' 

Language^ 


Ethnicity^ 


Population 

Density'' 


Mean ES 


1 1 


3 


5 


1.02 


1 1 


2 


2 


1.69 


1 1 


4 


2 


0.18 


1 2 


4 


3 


0.96 


2 1 


4 


5 


0.18 
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Chapter 5 

IMPACT OF HOME AND PARENT PROGRAMS ON YOUNG 
CHILDREN'S EARLY LITERACY SKILLS 



Christopher J. Lonigan 
Florida State University 

with 

Kathy Escamilla 
University of Colorado 

Dorothy Strickland 
Rutgers University 

and 

The National Early Literacy Panel 



Some intervention studies that the National Early Literacy Panel (NELP) examined focused 
on the evaluation of parent and home programs that were aimed at improving young children’s 
preliteracy and literacy skill development, findings from descriptive and correlational research 
studies have often been interpreted as demonstrating a link between supportive parental 
involvement (PI) and children’s early literacy-related development. Por instance, some research 
studies (e.g., Hart & Risley, 1995; Snow, Barnes, Chandler, Goodman, & Hemphill, 1991) 
suggest that children from homes in which parents engage them in elaborated conversations, 
model the uses of literacy, and engage them in activities that promote basic understandings 
about literacy (e.g., shared book reading) will have better-developed language and literacy 
skills (including the kinds of predictor skills identified in Chapter Two of this report) than will 
children from homes in which these activities are less frequent. Several national efforts, such as 
Reading Is fundamental and Reach Out and Read, have focused on getting books into the hands 
of parents and children and promoting regular parent-child book reading. These and other efforts 
have worked to some extent, as shown in national surveys indicating an increase in parent-child 
literacy activities among families with preschoolers (Tabors, Snow, & Dickinson, 2001). 

Some educators consider parent education an integral component of early childhood programs; 
however, reports of their effectiveness have varied widely. Many of the studies reviewed in this 
chapter were initiated with the assumption that successful PI programs help parents understand 
the importance of their role as first teachers and equip them with both the skills and the 
strategies to foster their children’s language and literacy development. The purpose of this 
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research summary was to determine the extent to which these interventions that used parents, 
typically mothers, as the primary agent of intervention actually confer a benefit to the children in 
terms of their literacy development. 

As previously described, to be included in NELP s analyses of intervention effects, studies need to 
(1) use a group-comparison design (randomized control trial [RCT] or quasiexperimental design 
[QED]), (2) use outcome measures that were either assessments of a conventional literacy skill 
(i.e., decoding, reading comprehension, spelling) or assessments of one of the skills identified as a 
predictor of later literacy skills by NELP’s analyses of predictive studies, and (3) report sufficient 
information to allow an effect size (ES) to be calculated. Additionally, studies that used a QED 
were required to have evidence of initial group comparability (i.e., the groups had pretest scores 
within 0.5 standard deviation of each other). A total of 23 studies were identified that met these 
criteria and that examined the impacts of home and parent early literacy programs. 



Overall Estimates of Intervention Impacts 

Across NELP’s analyses of intervention effects, nine categories of dependent variables were 
considered. The results related to the overall impacts of this group of intervention studies 
across all outcome variables are presented in Table 5.1. The data reported in the table include 
the outcome variables presented in alphabetical order, the numbers of studies that contributed 
to the effect estimates, an estimate of each ES based on a fixed-effect model, an estimate of 
each ES based on a random-effect model,' the 95 percent upper and lower bounds of the ESs 
based on the random-effect model, and the statistical significance of the ESs from the random- 
effect model. Most of the studies in this category examined the impacts of the interventions 
on outcome variables reflecting oral language skills (18 studies) or general cognitive ability (six 
studies; included because of its relationship to IQ, which was one of the predictor variables 
identified in the earlier analysis). Eor eight of the other 10 categories of outcome variables, only 
one or two studies included outcomes for the variable. No study examined the impact of home 
or parent programs on rapid naming or on visual processing as outcome variables. 

As can be seen in Table 5.1, home and parent programs had statistically significant effects on 
measures of oral language (small) and cognitive ability (moderate to large). There were two other 
statistically significant effects of home and parent programs (i.e., memory, writing); however, 
each of these effects was based on a single study, which represents too few studies to allow 
unambiguous interpretation. Examination of the confidence intervals (CIs) for the oral language 
and cognitive ability ES estimates shows that they were overlapping. Hence, the effects of home 
and parent programs were statistically equivalent on these two outcomes. Overall, the results 
reported in Table 5.1 indicate that home and parent intervention programs included in these 
studies had a statistically significant and positive impact both on young children’s oral language 
skills and general cognitive abilities. 



‘ Except where noted, all ES estimates and associated statistics are based on a random-effect model. 
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Table 5. 1. Estimates of Effect Sizes Across Outcomes for Home and Parent Literacy Programs for Each 
Dependent Variable 



Dependent Variable 


Fixed 

ES 


Random 

ES 


95% Cl 


Noi 

Studies 


P 

for ES 








Lower 


Upper 












Bound 


Bound 






AK 


-0.03 


-0.03 


-0.31 


0.24 


1 


0.81 


Cognitive ability 


0.65 


0.92 


0.22 


1.62 


6 


0.01 


Memory 


1.17 


1.17 


0.50 


1.84 


1 


0.0006 


Oral language 


0.28 


0.37 


0.18 


0.55 


18 


0.0001 


PA 


0.22 


0.21 


-0.12 


0.54 


2 


0.21 


Reading 


0.28 


0.28 


-0.12 


0.68 


1 


0.17 


Reading readiness 


-0.05 


0.05 


-0.33 


0.22 


1 


0.71 


Spelling 


0.09 


0.09 


-0.18 


0.37 


1 


0.51 


Writing 


0.52 


0.52 


0.23 


0.81 


1 


0.0005 



Note: Cl = Cl for random-effect model. 



Estimates of Intervention Impacts by Characteristics of Interventions 
and Populations 

In addition to the analyses of the overall effects of home and parent programs on children’s 
literacy development, analyses were conducted that addressed questions about the effects of 
different features of the programs and how these home and parent programs differed in their 
impacts on children’s skills across different population characteristics of the samples included in 
the studies. These estimations of effects by characteristics of the intervention or the population 
could be conducted only in cases in which sufficient numbers of studies used a specific outcome 
measure. Given that these subanalyses involved two or more categories, only studies that 
included oral language as an outcome measure could be included in these analyses, to maximize 
the likelihood that each subcategory would have sufficient studies (i.e., three or more) to allow 
interpretation. With only six studies that examined outcomes in the cognitive ability domain, 
the likelihood that divisions of these studies would yield three studies per category or that the 
classifications of studies would be independent of each other was low. 

The first subanalysis was conducted to determine whether there were any differences in ES 
estimates based on the type of study design (RCT versus QED). The results of the analysis for 
oral language outcomes are shown in Table 5.2. Although there was a trend for studies using a 
nonrandomized study design to produce larger results than did experiments, this result was not 
statistically reliable (Q[l, 16] = 1.82,^= 0.18). 

NELP’s analysis of predictors of conventional forms of reading and writing growth revealed 
that some oral language skills were better predictors of later reading outcomes than others. 
Specifically, measures of more complex oral language skills (such as grammar, the ability to 
define words, and listening comprehension) and composite measures that included these skills 
were stronger predictors of later reading skills than were measures of simple vocabulary. For this 
reason, the panel considered whether there was a different impact of home and parent programs 
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on composite measures of oral language from that on measures of simple vocabulary. The 
results of this analysis are shown in Table 5.3. Although the ES estimate for measures of simple 
vocabulary was larger than that for composite measures of oral language, this difference was not 
statistically reliable. The CIs for the ES estimates overlapped. 



Table 5.2. Impact of Home and Parent Programs on Oral Language Outcome by Nature of Intervention 
in Study 



Design of Study 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








RCT 


0.28 


0.10 


0.09 


0.46 


2.85 


13 


0.012 


QED 


0.51 


0.14 


0.23 


0.79 


3.54 


5 


0.003 



Note: ESs based on random-effect model 



Table 5.3. Effect Sizes of Home and Parent Programs on Measures of Simple Vocabulary and Composite 
Measures of Oral Language 



Measure 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Vocabulary 


0.41 


0.15 


0.13 


0.70 


2.81 


8 


0.005 


Composite 


0.27 


0.16 


-0.04 


0.58 


1.72 


6 


0.086 



Note: ESs based on random-effect model 



Analysis of Intervention Effects by Ages of Children 

The studies involving the impacts of home and parent intervention programs could have 
included a broad age range of children. In fact, only one study included children who were 
already kindergarten age at the time of the intervention. However, there were sufficient numbers 
of studies to allow a comparison of children across the ages of 0 to 3 and 4 to 5. Results of this 
analysis are shown in Table 5.4. There was no statistically reliable difference in ESs for studies 
involving younger children and ESs for studies involving older children (Q[l, 16] = 0.26, 
p^O.61). 



Table 5.4. Impact of Home and Parent Interventions on Oral Language Outcome by Ages of Children in Study 



Age of Children 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Zero to three 


0.42 


0.14 


0.14 


0.69 


2.96 


1 


0.009 


Three to five 


0.32 


0.12 


0.09 


0.56 


2.71 


11 


0.015 



Note: ESs based on random-effect model 
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Analysis of Intervention Effects by Type of Intervention 



The 1 8 studies that included oral language as an outcome were diverse in the focus, content, 
and duration of intervention studied. Two studies examined the effect of training parents to use 
dialogic reading (DR) (see Chapter Four). Six studies used a home visiting program to either 
teach parents general stimulation activities for their children or teach parents more focal oral 
language stimulation activities. Five additional studies taught parents similar general stimulation 
or language interaction strategies in a university or clinic setting. One of these studies was the 
Abecedarian project, in which parents received training and support for more than four years. 
Two studies taught parents to act as speech-language clinicians for their children with speech- 
language disorders. Two studies investigated the impacts of having parents engage in activities 
coordinated with activities occurring in their children’s kindergarten or preschool. Finally, one 
study examined the impact of an intervention program that included both parent training and 
weekly parent-child sessions at the children’s preschool. 

Given the variability in the types of interventions (e.g., from general stimulation programs for 
infants to parents acting as speech-language therapists for their children with speech-language 
disorders) as well as the relatively low number of studies in this group, it was difficult to 
identify meaningful subgroups of studies to examine possible moderators of ES estimates. The 
studies included in this group are shown in Appendix 5. A. With few exceptions, there was not 
significant variability among the observed ESs for the studies. More than half of the studies 
yielded moderate to large positive ESs. Interventions in the six studies that yielded near zero to 
negative ESs seemed not to share any obviously meaningful characteristic. One of the studies 
was the Abecedarian project (e.g., Roberts, Rabinowitch, Bryant, Burchinal, Koch, & Ramey, 
1989), which included one of the more focused and intensive parent interventions. One of the 
studies examined the effects of a general home-visiting program by paraprofessionals and nurses 
(Olds, 2002). One study examined the effects of teaching parents to encourage and support 
children’s narratives (Peterson, Jesso, & McCabe, 1999). Two studies concerned the impact of 
parents acting as intervention agents for their children with speech-language disorders (Eiserman, 
McCoun, & Escobar, 1990; Tannock, Girolametto, & Siegel, 1992), and one study examined the 
impact of adding a parent-based intervention component to a center-based program (Innocenti, 
Hollinger, Escobar, & White, 1993). 

Studies were classified according to whether and for whom materials were provided as a part of 
the program. Table 5.5 summarizes this information for the 14 studies in which this information 
could be coded. Although there was a trend for studies that did not provide materials as part of 
the program to yield smaller ESs than programs that provided materials for parents, children, or 
both, these differences were not statistically reliable (Q[3, 10] = 1.66, p = 0.65). Moreover, the 
Cl for the ES estimate for studies in which no materials were provided overlapped with those for 
studies that provided materials to parents, children, or both. 
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Table 5.5. Impact of Home and Parent Programs on Oral Language Outcome by Provision of Materials as 
Part of Intervention 



Materials Provided 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








None 


0.26 


0.21 


-0.16 


0.67 


1.22 


4 


0.249 


For parent and child 


0.53 


0.18 


0.18 


0.88 


3.00 


4 


0.013 


For child only 


0.49 


0.21 


0.07 


0.91 


2.30 


3 


0.044 


For parent only 


0.66 


0.26 


0.15 


1.18 


2.51 


3 


0.031 



Note: ESs based on random-effect model 



Analysis of Intervention Effects by Demographics of Study Samples 

For all analyses of intervention effects, NELP examined variations in estimated ESs for a 
common set of demographic variables. The demographic variables in these analyses included the 
socioeconomic status (SES) of the families of the children included in the study samples, ethnic 
classification of study participants, and the population density of the location where the study 
was conducted (i.e., rural, urban, suburban, mixed, unknown). In many of the studies included 
in the summary of home and parent programs, either these demographic characteristics were not 
reported or the samples used in the studies represented a mix of the demographic categories. 

A summary of the results of the analyses for ESs based on the SES of the study samples in home 
and parent programs is shown in Table 5.6. ES estimates did not vary significantly as a function 
of the SES classification of the study samples for the oral language outcome (Q[2, 13] = 0.40, 
p = 0.82). The majority of studies included populations of children who were classified as low 
SES. A summary of results of the analyses for ESs for home and parent programs based on the 
ethnicity of the study sample is shown in Table 5.7. In this analysis, the ES estimates did not vary 
significantly as a function of ethnicity (Q[2, 15] = 2.89, p = 0.24). ESs within each classification 
were statistically significant; however, the majority of study samples were classified as mixed 
ethnicity or unknown ethnicity. Finally, a summary of the results for the analyses of ESs for 
home and parent programs based on population density is shown in Table 5.8. In this analysis, 
ESs did vary significantly as a function of population density (Q[3, 14] = 1 1.73, p = 0.008). 
Examination of the individual groups, however, indicated that this significant variability was 
between the ES estimate for studies conducted in rural areas and that in all other population- 
density classifications (i.e., the Cl for the rural classification did not overlap those for any of the 
other classifications). However, the ES estimate for the rural classification was based on a single 
study, so this significant result is not an interpretable finding. 
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Table 5.6. Subanalysis for Oral Language Outcomes by Socioeconomic Status Classification of 
Study Sample 



SES Classification 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Low 


0.38 


0.11 


0.16 


0.59 


3.43 


12 


0.005 


Not low 


0.19 


0.29 


-0.38 


0.76 


0.65 


3 


0.528 


Unknown 


0.41 


0.29 


-0.17 


0.98 


1.38 


1 


0.190 


Note: ESs based on random-effect model. 














Table 5.7. Subanalysis for Oral Language Outcomes by Ethnicity of Study Sample 






Ethnicity 


Means 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Caucasian 


0.72 


0.28 


0.19 


1.26 


2.63 


2 


0.019 


African American 


0.48 


0.20 


0.09 


0.87 


2.42 


4 


0.029 


Mixed or unknown 


0.28 


0.09 


0.09 


0.46 


2.93 


12 


0.010 


Note: ESs based on random-effect model. 














Table 5.8. Subanalysis for Oral Language Outcomes by Population Oensity of Study Sample 




Group 


Mean ES 


SE 


95% Cl 




t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Rural 


1.53 


0.37 


0.81 


2.26 


4.13 


1 


0.001 


Urban 


0.22 


0.09 


0.04 


0.41 


2.37 


8 


0.033 


Suburban 


0.33 


0.15 


0.04 


0.63 


2.20 


2 


0.045 


Unknown 


0.32 


0.12 


0.09 


0.55 


2.68 


7 


0.018 



Note: ESs based on random-effect model 



Summary and Conclusions 

Results from this meta- analysis of the impacts of home and parent programs on the literacy 
skills of young children indicate that these interventions yield a moderate to large effect on oral 
language outcomes and general cognitive abilities. These effects appear to be robust to variations 
in childrens ages and demographic characteristics of families. Additionally, the effects of these 
programs on childrens oral language skills were consistent across measures of simple vocabulary 
and measures of more complex oral language skills. Although home and parent programs could 
impact other aspects of literacy, only a handful of studies included these other outcomes, and no 
other outcome was included in more than two of these studies (for example, alphabet knowledge 
[AK] was included in only one study, and phonological awareness [PA] in only two). Therefore, 
it was not possible to determine whether there were other effects of home and parent programs; 
consistent with other conclusions in this report, it would seem prudent at this time to rely on 
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parent programs to address only outcomes that such programs have been proven to stimulate and 
to employ such interventions in combination with other approaches that address other issues. 
Future research may be able to expand our understanding of the effectiveness of such parent and 
home approaches. Also, at this time, there are too few studies to allow fine-grained analysis of the 
impact of population variables, such as SES, ethnicity, or population density. Given the existing 
studies, however, there is no evidence that sample characteristics moderate the effects of home 
and parent interventions programs and no reason to withhold such programs from particular 
subgroups of children. 

The commonality across all of the programs examined by this group of studies is that they 
somehow involved parents as the agents of intervention for children. Nevertheless, these 
programs varied greatly in potentially important ways. For example, some of the programs 
had more general goals (such as trying to improve childrens health, behavior, or cognitive 
functioning); others aimed at more specific literacy goals (such as improving language skills). 
Because of the great amount of variation evident in these approaches, it is not yet possible to 
point to one or two examples of replicated models of successfully involving parents in enhancing 
their childrens developmental outcomes. Additional research on identification of key aspects of 
home and parent programs is needed. 

It was not possible to examine the question of the additive effects of home and parent programs in 
the context of high-quality center-based education programs. A few of the studies contrasted the 
effects of PI combined with an early childhood program with early childhood programs alone. In 
some cases, there was an additive effect of the parent program, and, in some cases, there was not. 
Many of the interventions included in this group of studies involved frequent home visits or one- 
on-one parent-training sessions. With the growing availability of universally available, federal- or 
state-funded early childhood education programs, understanding the impact of home and parent 
programs in the context of high-quality early childhood education deserves attention. 

Ultimately, attention to the nature, quality, and scope of home and parent intervention programs 
is required to identify those likely to be successful and those less likely to be successful. In the 
majority of studies examined in this meta-analysis, the interventions were delivered to parents 
by the developers of the intervention or by those who were supervised closely by the developers. 
Whether such interventions could be taken to scale — implemented broadly by individuals with 
limited or no contact with the developers — is yet unknown. 

Finally, it is important to recognize that none of the more commonly used programs of 
enhancing PI in young childrens literacy development (e.g., Chicago Child- Parent Centers, 
Parents as Teachers) was evaluated in the set of studies reviewed. Consequently, the results of 
this meta- analysis do not confirm effectiveness of these specific programs. Notably, only one 
study included in the analysis involved the typical model in which parent education, parenting 
education, and parent-child time was evaluated (McQueen & Washington, 1988). Whereas 
this study yielded a moderate ES (0.74), the degree of PI was relatively intensive. In addition 
to participating in parent education and parenting classes, each mother worked as a teaching 
assistant in her child’s classroom. In this context, the program was effective. Knowing whether 
all of these components and this level of intensity are required to achieve a positive outcome are 
questions that need to be addressed by future studies. 
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Appendix 5.A. Primary Coding of Home and Parent Program Studies with 
Oral Language Outcome Measures 



Study 


Type of 
Study’’ 


Child 

Age*' 


Material” 


Barnett, W. S., Escobar, C. M., & Ravsten, M. T. (1988). Parent 
and clinic early intervention for children with language handicaps: 
A cost-effectiveness analysis. Journal of the Division for Early 
Childhood, m), 290-298. 


1 


1 


4 


Crain-Thoreson, C., & Dale, P. S. (1999). Enhancing linguistic 
performance: Parents and teachers as book reading partners for 
children with language delays. Topics in Early Childhood Special 
Education, /S(1), 28-40. 


1 


0 




Cronan, T. A., Brooks, L. B., Kilpatrick, K., Bigatti, S. M., & Tally, S. 
(1999). The effects of a community-based literacy program: One- 
year follow-up findings. Journal of Community Psychology, 27(4), 
431-442. 


2 


1 


3 


Donachy, W. (1976). Parent participation in pre-school education. 
British Journal of Educational Psychology, 46, 31-39. 


2 


0 


4 


Eiserman, W. D., McCoun, M., & Escobar, C. M. (1990). A 
cost-effectiveness analysis of two alternative program models 
for serving speech-disordered preschoolers. Journal of Early 
Intervention, ?4(4), 297-317. 


1 


0 


3 


Gray, S. W., & Ruttle, K. (1 980). The family-oriented home visiting 
program: A longitudinal study. Genetic Psychology Monographs, 
/02, 299-316. 


1 


1 


3 


Innocent!, M. S., Elollinger, P. D., Escobar, C. M., & White, K. 

R. (1993). The cost-effectiveness of adding one type of parent 
involvement to an early intervention program. Early Education and 
Development, 4(4), 306-326. 


1 


0 


1 


Jordan, G. E., Snow, C. E., & Porche, M. V. (2000). Project EASE: 
The effect of a family literacy project on kindergarten students' 
early literacy skills. Beading Research Quarterly, 35(4), 524-546. 


2 


0 


2 


Klein, P. S., & Alony, S. (1 993). Immediate and sustained effects of 
maternal mediating behaviors on young children. Journal of Early 
Intervention, 17{2], 177-193. 


1 


1 


— 


Levenstein, P. (1970). Cognitive growth in preschoolers 
through verbal interaction with mothers. American Journal of 
Orthopsychiatry, 40(3), 426-432. 


2 


0 


2 
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SES-* 


Ethnicity‘S 


Population 

Density' 


ES 


2 


4 


5 


0.81 




4 


5 


0.29 


1 


4 


5 


0.58 


— 


4 


2 


0.54 


2 


1 


3 


0.04 


1 


4 


5 


0.83 


1 


4 


2 


- 0.03 


3 


4 


3 


0.41 


1 


4 


2 


0.51 


1 


2 


2 


0.41 
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Study 


Type of 
Study’’ 


Child 

Age*' 


Material” 


McQueen, A. B., & Washington, V. (1 988). Effect of intervention 
on the language facility of poor, black adolescent mothers and 
their preschool children. Early Child Development and Care, 33, 
137-152. 


1 


0 


2 


Olds, D. L., Robinson, J., O'Brien, R., Luckey, D. W., Pettit, L. M., 
Elenderson, C. R., Ng, R. K., Sheff, K. L, Korfmacher, J., Eliatt, 

S., & Talmi, A. (2002). Elome visiting by paraprofessionals and 
by nurses: A randomized, controlled trial. Pediatrics, 1 10{3], 
486-496. 


1 


1 




Peterson, C., Jesso, B., & McCabe, A. (1999). Encouraging 
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“ Design: 1 = FtCT, 2 = QED. 
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' Material: 1 = No material provided, 2 = material provided for parent and child, 3 = material provided for child only, 4 = material provided for parent only. 
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' Population density: 1 = Rural, 2 = urban, 3 = suburban, 4 = mixed, 5 = unknown. 
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SES-* 


Ethnicity‘S 


Population 

Density' 


ES 


1 


2 


2 


0.74 


1 


4 


2 


0.05 


1 


4 


5 


- 0.57 


1 


2 


5 


0.07 


1 1 1 1.53 


1 


2 


2 


0.72 


0.73 


0.71 


2 


4 


2 


- 0.11 


1 


4 


5 


0.18 



DEVELOPING EARLY LITERACY: REPORT OP THE NATIONAL EARLY LITERACY PANEL 



187 



References 



Eiserman, W. D., McCoun, M., & Escobar, C. M. (1990). A cost-effectiveness analysis of 
two alternative program models for serving speech-disordered preschoolers. Journal of Early 
Intervention, 14{A), 297—317. 

Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experience of young 
American children. Baltimore: Paul H. Brookes Publishing. 

Innocenti, M. S., Hollinger, P. D., Escobar, C. M., & White, K. R. (1993). The cost-effectiveness 
of adding one type of parent involvement to an early intervention program. Early Education 
and Development, 4{A), 306—326. 

McQueen, A., & Washington, V. (1988). Effect of intervention on the language facility of poor, 
black adolescent mothers and their preschool children. Early Child Development and Care, 33, 
137-152. 

Olds, D. L., Robinson, J., O’Brien, R., Luckey, D. W, Pettitt, L. M., Henderson, C. R., et al. 
(2002). Home visiting by paraprofessionals and by nurses: A randomized, controlled trial. 
Pediatrics, llCKf), 486— 497. 

Peterson, C., Jesso, B., & McCabe, A. (1999). Encouraging narratives in preschoolers: An 
intervention study. Journal of Child Language, 26{l), 49-67. 

Roberts, J. E., Rabinowitch, S., Bryant, D. M., Burchinal, M. R., Koch, M. A., & Ramey, C. T. 
(1989). Language skills of children with different preschool experiences. Journal of Speech & 
Hearing Research, 32, 773-800. 

Snow, C. E., Barnes, W. S., Chandler, J., Goodman, I. E, & Hemphill, L. (1991). Unfulfilled 
expectations: Home and school influences on literacy. Cambridge, Mass.: Harvard University Press. 

Tabors, P. O., Snow, C. E., & Dickinson, D. K. (2001). Homes and schools together: 

Supporting language and literacy development. In D. K. Dickinson & P. O. Tabors (Eds.), 
Beginning literacy with language: Young children learning at home and school (pp. 3 1 3-334). 
Baltimore: Brookes. 

Tannock, R., Girolametto, L., & Siegel, L. S. (1992). Language intervention with children who 
have developmental delays: Effects of an interactive approach. American Journal on Mental 
Retardation, 97{2), 145—160. 



188 



DEVELOPING EARLY LITERACY: REPORT OF THE NATIONAL EARLY LITERACY PANEL 



Chapter 6 

IMPACT OF PRESCHOOL AND KINDERGARTEN PROGRAMS 
ON YOUNG CHILDREN'S EARLY LITERACY SKILLS 



Victoria Molfese 
University ofLouisviiie 

Laura Westberg 
Nationai Center for Famiiy Literacy 

with 

The National Early Literacy Panel 



A variety of early childhood programs have been studied since the early 1 960s to determine their 
effectiveness in improving social and academic outcomes for young children. For example, Perry 
Preschool Project (Schweinhart, Monde, Xiang, Barnett, Belfield, & Notes, 2005; Schweinhart 
& Weikart, 1980, 1997; Weikart, Bond, & McNeil, 1978) and the federally funded program 
Head Start, along with a variety of state preschool programs, have been the focus of research 
(Gilliam & Zigler, 2000) (as have other programs, such as the Abecedarian project [Ramey & 
Campbell, 1984], the Chicago Child- Parent Center [Reynolds, 1999], and a plethora of early 
prevention efforts) (Nelson, Westhues, & MacLeod, 2003). All of these programs have focused 
on children in poverty or who are at risk for educational or social failure, and they have included 
a broad range of program services, including education, nutrition, health and social services, 
home-visiting interactions, and parent support. Provision of services in these programs targeted 
varied ages, including infants and preschoolers who were then followed into kindergarten, early 
elementary school, and beyond. 

The National Early Literacy Panel (NELP) examined the effectiveness of several of such 
preschool and kindergarten programs and interventions aimed at the development of early 
literacy and conventional literacy skills.^ (Unfortunately, the studies of some of the widely known 
programs have either not been reported in refereed journals or have not focused on literacy- 
learning outcomes, so they could not be examined here). The panel set out to determine whether 
such programs confer children with an advantage in literacy learning or in the development 
of early skills that predict later literacy success. The studies included in this chapter met the 
selection criteria established by the panel for the meta-analysis including (1) group design using 

‘ Several studies that examined code-focused interventions (see Chapter Three) could have been double counted, since many of 
those studies were carried out within preschools and kindergartens. Some of the book-sharing (Chapter Four) and language 
intervention studies (Chapter Seven) could have been included here as well. However, given the sufficient numbers of such 
studies in the other sets, the panel members believed it best to consider them separately, so this chapter considers school-based 
interventions that could not accurately be included in those other chapters. 
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either a randomized control trial (RCT) or a quasiexperimental design (QED) with initial group 
equivalency, (2) an intervention that measured effectiveness on early literacy or conventional 
literacy skills, and (3) sufficient data to calculate an effect size (ES). A total of 33 studies met 
these criteria. Ten of these studies evaluated the effectiveness of the Abecedarian project, and, 
since nine of these studies involved the same sample of children longitudinally, the results of 
these nine studies were combined and treated as a single group. 



Overall Estimates of Intervention Impacts 

Table 6.1 provides a summary of the overall effects of the various preschool and kindergarten 
interventions across all the different outcomes. The table includes, for each outcome variable (in 
alphabetical order), the number of studies that contributed to the ES estimates, an estimate of 
the ES based on a random-effect model,^ the 95 percent upper and lower bounds of the ESs, and 
the statistical significance of the ESs. The majority of the studies in this category provided effects 
for oral language (12 studies) and reading (nine studies). Fewer studies examined the impact of 
these interventions on alphabet knowledge (AK) (four studies), cognitive ability (four studies), 
readiness (three studies), spelling (three studies), phonological awareness (PA) (two studies), 
memory (two studies), print knowledge (two studies) and writing (two studies). Although 
cognitive ability per se did not arise in the predictor study, this variable is closely aligned with the 
various measures of IQ that were found to have predictive value in that earlier analysis. For that 
reason, the cognitive ability outcome is examined here. 

As indicated in Table 6.1, preschool- and kindergarten-based interventions resulted in large, 
statistically significant outcomes for readiness measures ( 1 .23) and small to moderate effects on 
spelling measures (0.34). Although statistically significant effects also were found for memory 
(0.47) and print knowledge (1.00), these outcomes were measured in too few studies to allow for 
a reliable determination of the impact of preschool and kindergarten experiences on these skills. 

It should be noted that readiness tests do not represent a single skill; they are composite measures 
encompassing many early literacy predictors, including AK, concepts of print, vocabulary, 
memory, and PA. 

Although the average ESs for preschool and kindergarten programs were large enough to be of 
educational importance for several literacy variables (such as reading, writing, and AK), these 
differences did not reach statistical significance for the small numbers of studies combined in 
these analyses. Perhaps as more studies are completed with these kinds of outcomes, it would 
be possible to conclude that kindergarten and preschool interventions have a general ability 
to improve student literacy performance. However, the oral language outcomes were both 
statistically insignificant and so small as to be of questionable importance, though preschool 
and kindergarten efforts with a more explicit focus on oral language development may have very 
different results. 



Except where noted, all ES estimates and associated statistics are based on a random-effect model. 
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Table 6. 1. Estimates of Effect Sizes Across Domains for Interventions Involving Preschool and Kindergarten 
Programs for Each Dependent Variable 



Dependent Variable 


Fixed ES 


Random ES 


95% Cl 


Nof 

Studies 


P 

for ES 








Lower 

Bound 


Upper 

Bound 






AK 


0.31 


0.23 


-0.18 


0.64 


4 


0.27 


Cognitive ability 


0.30 


0.35 


-0.11 


0.80 


4 


0.13 


Memory 


0.47 


0.47 


0.15 


0.79 


2 


0.004 


Oral language 


0.10 


0.13 


-0.06 


0.31 


12 


0.17 


PA 


0.08 


0.08 


-0.15 


0.31 


2 


0.49 


Print knowledge 


1.00 


0.98 


0.25 


1.70 


2 


0.008 


Readiness 


1.23 


1.22 


0.05 


2.38 


3 


0.04 


Reading 


2.05 


0.75 


-0.38 


1.89 


9 


0.19 


Spelling 


0.34 


0.34 


0.07 


0.60 


3 


0.01 


Writing 


0.72 


0.67 


-0.14 


1.48 


2 


0.11 



Note: Cl= Cl based on random-effect model. 



Estimates of Intervention Impacts by Demographic Characteristics of 
Study Samples 

NELP was interested in whether preschool and kindergarten programs had differential effects for 
children from different demographic groups, including socioeconomic status (SES), ethnicity, age 
of the children, and the population density of the location where the study was conducted (i.e., 
rural, urban, suburban, mixed or unknown). In many instances, these data were not reported in 
the research articles or the samples of children within the studies were mixed, resulting in too few 
studies to allow for interpretation. 

Socioeconomic Status 

Half of the studies with oral language outcomes were conducted with children of low SES {n = 

6). Only one other study with oral language outcomes reported SES of the children, and that 
included a mixed sample, which makes it impossible to interpret this effect. Likewise, even 
fewer studies with reading outcomes reported SES information (n = 4). Although the large 
effect reported for reading outcomes in the one study with mixed-SES children was significantly 
different from that for the other groups (Q [2, 1] = 54.40, p < 0.001), the limited number of 
studies (« < 3) contributing to this effect does not allow for meaningful interpretation. The 
results of the SES analysis are included in Table 6.2. 
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Table 6.2. Effect Sizes for Oral Language and Reading Outcomes for Preschool and Kindergarten Program 
Interventions Based on Socioeconomic Status of Study Sample 




2.56 6 0.05 

1.18 2 0.45 

1.08 1 0.47 

1.83 1 0.13 

8.85 1 0.07 



Low SES 


Oral language 


0.32 


0.12 


0.08 


0.56 


Reading 


0.08 


0.07 


-0.05 


0.21 


Not low SES 


Oral language 


— 


— 


— 


— 


Reading 


0.27 


0.25 


-0.21 


0.75 


Mixed SES 


Oral language 


0.22 


0.12 


-0.02 


0.45 


Reading 


1.19 


0.39 


0.92 


1.45 



Age 

Many more studies with oral language outcomes included prekindergarten than included 
kindergarten children. The reverse was true for studies with reading outcomes, as only two of the 
nine studies include children younger than kindergarten age. The results of the age subanalysis 
are included in Table 6.3. TUthough the ES estimate for interventions with reading outcomes 
that included kindergarten children was larger than that for prekindergartners, this effect was 
not statistically reliable (Q[l,7] = 0.68, p = 0.41). And while the comparison of interventions 
with oral language outcomes by age was statistically significant (Q[l,10] = 6.18, /> = 0.01), the 
confidence intervals (CIs) overlapped, indicating that there was no evidence that age moderated 
the impact of the preschool and kindergarten programs on oral language outcomes. 



Table 6.3. Effect Sizes for Oral Language and Reading Outcomes of Preschool and Kindergarten Program 
Interventions Based on Age of Study Sample 



Age 


Outcome 


Mean ES 


SE 


95% Cl 




t 


n 


P 










Lower 

Bound 


Upper 

Bound 








Prekindergarten 


Oral language 


-0.03 


0.08 


-0.20 


0.13 


-0.44 


9 


0.67 


Reading 


0.33 


0.59 


-0.83 


1.49 


0.55 


2 


0.60 


Kindergarten 


Oral language 


0.28 


0.09 


0.09 


0.46 


2.95 


3 


0.01 


Reading 


0.88 


0.32 


0.21 


1.31 


2.79 


7 


0.03 



Ethnicity 

Overall, there were insufficient numbers of studies of either oral language or reading outcomes to 
determine whether childrens ethnicity had any impact on the effectiveness of the interventions. 
For preschool and kindergarten programs with reading outcomes, only two groups could be 
compared, and the majority of these studies included mixed samples of children. The results 
of this subanalysis, shown in Table 6.4, indicate no statistically significant differences between 
these two groups (Q[l,7] = 0.36, p = 0.55). Moreover, with only one study including a single 
ethnic group, there is no way to discern whether the effects of ethnicity moderated the impact 
of preschool and kindergarten programs on reading outcomes. Although there were more ethnic 
groups represented in the preschool and kindergarten program interventions with oral language 
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outcomes, the majority of these studies included mixed samples of children, and only one study 
each using Caucasian, African American, and Hispanic populations. There were no statistically 
significant differences among these groups (Q[3, 8] = 2.99, p = 0.39), and, because of the limited 
number of studies for these ethnic classifications, the estimated ESs were not interpretable. 



Table 6.4. Effect Sizes for Oral Language and Reading Outcomes for Preschool and Kindergarten Program 
Interventions Based on Ethnicity of Study Sample 



Ethnicity 


Outcome 


Mean ES 


SE 


95% Cl 




t 


n 


P 










Lower 


Upper 
















Bound 


Bound 








Caucasian 


Oral language 


-0.58 


0.46 


-1.47 


0.31 


-1.27 


1 


0.24 


Reading 


0.27 


0.87 


-1.41 


1.96 


0.31 


1 


0.77 


African 


Oral language 


0.07 


0.36 


-0.64 


0.79 


0.20 


1 


0.85 


American 


Reading 


— 


— 


— 


— 


— 


— 


— 


Hispanic 


Oral language 
Reading 


0.38 


0.33 


-0.27 


1.02 


1.13 


1 


0.29 


Mixed 


Oral language 


0.13 


0.09 


-0.05 


0.31 


1.46 


9 


0.18 


Reading 


0.82 


0.30 


0.23 


1.41 


111 


8 


0.03 



Population Density 

The majority of studies with oral language and reading outcomes did not provide sufficient 
information to classify where the study was conducted according to population density (rural, 
urban, suburban, or mixed). For preschool and kindergarten program interventions with oral 
language outcomes, the subanalysis included four studies classified as urban and eight studies 
classified as unknown (see Table 6.5). No study with oral language outcomes was classified as 
rural, suburban, or mixed, causing too few instances to generate reliable ES estimates. Differences 
in the ES estimates were not statistically significant (Q[l,9] = 0.19, p = 0.19). Studies that 
included preschool and kindergarten program interventions with reading outcomes had too 
few studies classified to interpret the impact of population density. The differences were not 
statistically reliable (Q[2,5] = 0.94, p = 0.63). 



Table 6.5. Effect Sizes for Oral Language and Reading Outcomes for Preschool and Kindergarten Program 
Interventions Based on Population Oensity of Study Sample 



Population n ^ 

„ Outcome 


Mean ES 


SE 


95% Cl 


t 


n 


p 


Density 















Lower Upper 

Bound Bound 



Urban 


Oral language 
Reading 


0.18 

0.78 


0.13 

0.61 


-0.08 

-0.41 


0.45 

1.97 


1.37 

1.29 


4 

2 


0.20 

0.25 




Oral language 


— 


— 


— 


— 


— 


— 


— 


Mixed 




Reading 


0.04 


0.84 


-1.62 


1.70 


0.04 


1 


0.96 




Oral language 


0.10 


0.13 


-0.15 


0.36 


0.78 


7 


0.45 


Unknown 




Reading 


0.94 


0.39 


0.18 


1.70 


2.43 


5 


0.06 
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Estimates of Intervention Impacts by Characteristics of Interventions 

The panel was interested in the characteristics of the interventions that moderate effects on early 
literacy and conventional literacy outcomes. To conduct analyses on these moderator variables, a 
sufficient number of studies had to have the same outcomes. Only two outcomes, oral language 
and reading, had sufficient, though small, numbers of studies to make these comparisons. 

RCTs and QEDs (with pretest equivalency of the comparison groups) were used to determine 
the impact of preschool and kindergarten interventions on early literacy skills and conventional 
literacy. A subanalysis comparing the experimental and quasiexperimental studies was conducted 
to ascertain the effects of study design of these interventions on oral language and reading 
outcomes. The results of this analysis are included in Table 6.6. Four experimental studies 
were compared to eight quasiexperimental studies with oral language outcomes. There were no 
statistically significant differences in the estimates of ESs for these studies (Q[l,10], = 0.69). 

For reading outcomes, there were three experimental interventions and six quasiexperimental 
interventions. The difference in the estimated ESs for these studies was statistically significant 
(Q[T7] = 8.12, p = 0.004). However, further examination revealed that two of the three 
experimental studies involved teacher professional development (PD), whereas only one of the six 
quasiexperimental studies included this intervention. Therefore, the impact should be interpreted 
with caution because of the confounding of study type with teacher PD. 



Table 6.6. Effect Sizes for Oral Language and Reading Outcomes for Preschool and Kindergarten Program 
Interventions Based on Type of Study Oesign 



Study 

Design 


Outcome 


Mean ES 


SE 


95% Cl 




t 


n 


P 










Lower 

Bound 


Upper 

Bound 








RCT 


Oral language 


0.08 


0.14 


-0.19 


0.35 


0.60 


4 


0.56 


Reading 


1.60 


0.36 


0.89 


2.30 


4.44 


3 


0.003 


QED 


Oral language 


0.16 


0.09 


-0.08 


0.39 


1.32 


8 


0.22 


Reading 


0.33 


0.26 


-0.18 


0.84 


1.27 


6 


0.24 



When NELP examined the research to identify early skills that predicted later conventional forms 
of literacy, the complex forms of oral language skills (e.g., grammar and definitional vocabulary) 
were determined to be better predictors of later literacy than was simple vocabulary. Therefore, 
the panel examined whether there were differences in outcome from preschool and kindergarten 
programs when vocabulary and composite oral language were the outcome measures. Five studies 
assessed outcomes of simple vocabulary, and six considered composite measures of oral language; 
several studies used both vocabulary and composite oral language measures, so there is some 
overlap in the number of studies contributing to the estimate of ESs. Three studies were excluded 
from this analysis because they measured communication or oral language only and could not be 
categorized appropriately. Although the estimated ES for the composite measure was higher than 
that for the measures of vocabulary, this difference was not statistically reliable. The results of this 
analysis are included in Table 6.7. 
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Table 6.7. Effect Sizes of Preschool and Kindergarten Program Interventions on Measures of Simple 
Vocabulary and Composite Measures of Oral Language 



Measure 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Vocabulary 


0.13 


0.13 


-0.12 


0.38 


1.03 


5 


0.30 


Composite 


0.40 


0.15 


0.11 


0.68 


2.73 


6 


0.006 



Three characteristics of the various preschool and kindergarten program interventions were reported 
in sufficient detail in enough studies to allow them to be analyzed. Separate analyses by outcome 
variable were conducted for interventions that implemented literacy-focused curricula, PD for 
teachers, and parental involvement (PI) associated with the kindergarten or preschool program. 

Literacy-Focused Curricula 

Three studies examined the impact of the use of literacy-focused curricula in kindergarten 
classrooms, and all of these studies examined readiness as an outcome. Two of the interventions 
were implemented with at-risk kindergarten children and the third with preschoolers. One 
of the kindergarten interventions was a two-year study following children into first grade and 
measured achievement at the ends of the kindergarten and first-grade years. Children were 
assessed in kindergarten, and the results were used to identify children’s strengths and weaknesses 
for planning individual programs. Small groups of two to four children with similar needs 
were formed, and groups rotated through learning stations targeted to specific needs that were 
monitored by parent aides trained to work with the children. In addition to the learning stations 
that were stocked with educational materials to reinforce language, perceptual-motor, and 
number-concept skills, children were involved in a structured prereading program to teach letter 
names and letter sounds. Enrichment activities enhancing gross motor skills and music were also 
offered to kindergartners in the experimental group. 

The other kindergarten study was implemented with French-speaking children in Canada. The 
intervention in this study incorporated a multicomponent literacy program that included a 
well-stocked library corner and writing corner; opportunities to engage in reading and writing 
in the context of play; ffinctional reading of text and writing of children’s names; and daily story 
reading. In addition, a structured method (20 lessons) of PA training was used in a meaningful 
literacy context using nursery rhymes, writing, and word play. Along with the kindergarten 
literacy program, parents were invited to four 90-minute sessions at the beginning of the year 
to receive suggested activities and practical material. In half of the experimental classrooms, 
in addition to the basic literacy program, children received semiweekly PA sessions in which 
phonological activities were integrated into reading and writing events. Children participated in 
small groups of four to six for a total of 45 sessions, each 25 to 30 minutes long, over a period of 
eight months. Comparison children received the traditional kindergarten program consisting of a 
free-play curriculum, though they also received the 20 lessons in phoneme awareness (though not 
embedded in the experimental literacy context). 
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The third study was implemented with preschoolers, ages three to five years, in two child-care 
centers in an urban area. The intervention was structured to provide literacy enrichment by 
arranging the space for sustained play in literacy-relevant contexts (e.g., post office, library), with 
authentic literacy objects (e.g., cookbooks, recipe cards). Nonintervention classrooms made no 
changes to the classroom play environments. 

As indicated in Table 6.1, these studies of various literacy curricula had an average ES of 1.23, 
which is large and statistically significant. All three of these studies also examined oral language 
outcomes, so their results were compared with those of the other studies that had oral language 
outcomes but did not implement literacy-focused curricula. Results of this analysis are shown 
in Table 6.8. There was no statistically significant difference between studies that implemented 
literacy-focused curricula with oral language outcomes and those that did not (Q[l, 10] = 0.06 
;>= 0.81). 

Several studies with literacy-focused curricula measured the effects of those interventions on 
reading. One of the studies in this subcategory that was not discussed previously implemented 
the Reading Mastery (RM) curricula (Englemann & Bruner, 1995). These curricula, conducted 
with severely at-risk kindergarten through second-grade students, provided students with explicit 
instruction in phonemic awareness and decoding. Teachers received intensive in-service teacher 
training and year-long support within the schools in the form of feedback on program planning 
and implementation. The four studies with literacy-focused curricula that measured reading 
outcomes were compared to five studies with reading outcomes that did not implement literacy- 
focused curricula. The results of this comparison are shown in Table 6.9. There was a significant 
difference between the kindergarten programs with a literacy focus and those without a literacy 
focus (Q[l, 7] = 3.85 p = 0.05). further examination of these studies indicated that the literacy- 
focused curriculum interventions yielded an estimated mean ES between approximately three and 
a half to four times larger than those for studies that did not use a literacy-focused curriculum. 



Table 6.8. Effect Sizes for Oral Language and Reading Outcomes for Preschool and Kindergarten Program 
Interventions with Literacy-Focused Curricula 



Literacy 

Focus 


Outcome 


Mean ES 


SE 




95% Cl 


t 


n 


P 










Lower 

Bound 


Upper 

Bound 








No 


Oral language 


0.11 


0.11 


-0.12 


0.33 


0.94 


9 


0.37 


Reading 


0.33 


0.33 


-0.31 


0.97 


1.01 


5 


0.34 


Yes 


Oral language 


0.15 


0.14 


-0.13 


0.43 


1.06 


3 


0.31 


Reading 


1.29 


0.36 


0.58 


2.00 


3.56 


4 


0.009 



Professional Development 

PD of teachers was another characteristic of the interventions that was of interest for determining 
features that moderate the impact of preschool and kindergarten programs. Eor oral language 
outcomes, only two studies implemented PD with teachers, and 10 studies did not implement 
PD. Results of this comparison are shown in Table 6.9. There was no statistically significant 
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difference in the ES estimates between these groups (Q[l,10] = \.QA, p = 0.31); moreover, the 
small number of studies with PD {n < 3) in this comparison does not allow for interpretation. 

In contrast, there were enough studies to compare effects of PD in kindergarten programs on 
reading outcomes. Three studies implemented PD, and six studies did not. All three of the 
studies that implemented PD were included in the set of four studies that involved the impact 
of literacy-focused curricula on reading outcomes and have been described in previous analyses. 
Estimates of the effects of kindergarten programs with PD are shown in Table 6.9. The results of 
this comparison indicated a statistically significant difference in the groups on reading outcomes 
(Q[T 7] = 1420.23, p < 0.000). There was a stronger effect for the studies that implemented PD; 
however, it is likely that the estimation of the effects of PD in this analysis were not independent 
of the effects of literacy-focused curricula in these kindergarten programs, and, in combination, 
these characteristics of the interventions produced very strong effects on reading outcomes. 



Table 6.9. Effect Sizes for Oral Language and Reading Outcomes for Preschool and Kindergarten Program 
Interventions with Professional Oevelopment 




Oral language 0.18 0.10 -0.02 0.39 1.77 10 0.11 

Reading 0.17 0.06 0.06 0.28 3.00 6 0.02 

Oral language 0.002 0.14 -0.28 0.28 0.01 2 0.99 

Reading 2E5 0E3 2E9 2.72 81.42 3 0.000 



Parental Involvement 

Interventions could also be compared in terms of PI. Only three studies with oral language 
outcomes included some form of PI associated with the preschool or kindergarten program, 
and the three forms differed in their approaches. Two of the studies included prekindergarten 
children — one with very young low-income children and the other with prekindergarten children 
with disabilities. The third study involved kindergarten children of mixed income. The one 
study involving at-risk children included a language-enriched day care plus parent-education 
intervention and a parent intervention alone. Children entered the day-care program between 
the ages of six weeks and three months and attended five full days per week for 50 weeks per 
year until they entered kindergarten. The two treatment conditions were compared to a no- 
treatment control. The parent-education program included home visits every 1.5 weeks and 
parent-group meetings each month that focused on learning games, child management, and 
parent problem-solving regarding the child. Each family received a notebook of materials that 
contained developmental information, record-keeping materials, parent-skill information, and 
an index of learning activities. Another intervention involved a comparison of both low-income 
and high-income schools that received a number of high-quality childrens books plus teacher 
training with schools that received books only. Control schools did not receive books or training. 
In the training schools, teachers received 130 books for a parents’ lending library and parents 
attended two one-hour evening parent sessions and received a total of five copies of children’s 
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books. No further description of the parent sessions was reported. The third intervention in this 
subcategory compared a current classroom delivery system to the same system enhanced by one 
type of systematic PI. The classroom intervention included half-day, five-day-per-week services 
with large-group, small-group, and one-to-one individualized services for children with special 
needs. The PI intervention consisted of parent meetings organized around the Parents Involved 
in Education (PIE) curriculum (Pezzino & Lauritzen, 1986). PIE training modules were designed 
to provide parents with a systematic, conceptual, and hands-on experience in such areas as child 
development, observation and recording, targeting intervention behaviors, teaching processes, 
decisionmaking, and communicating with professionals. The training format included small 
groups of eight to 12 parents in lecture, discussion, and demonstrations for a total of 16 two- 
hour meetings once per week. In addition to the modules, there was a social-support component 
and practice at home. 

The three studies including PI were compared with nine studies that did not include PI for 
oral language outcomes. As seen in Table 6.10, the mean ES estimates for PI interventions 
and interventions without PI are similarly small, and differences between these groups were 
not statistically reliable (Q[l,10] = 0.00, p = 0.99). As shown in Table 6.10, no preschool and 
kindergarten interventions with reading outcomes included PI as a feature of the program. 
Therefore, no analysis could be conducted to discern these effects. 

Table 6. 10. Effect Sizes for Oral Language Outcomes for Preschool and Kindergarten Program Interventions 
with Parent Involvement 



PI 


Outcome 


Mean ES 


SE 


95% Cl 




t 


n 


P 










Lower 

Bound 


Upper 

Bound 








No 


Oral language 


0.13 


0.11 


CO 

CD 

CD 

1 


0.33 


1.17 


9 


0.27 


Reading 


— 


— 


— 


— 


— 


— 


— 


PI 


Oral language 


0.12 


0.16 


1 

CD 

CO 


0.43 


0.78 


3 


0.45 


Reading 


— 


— 


— 


— 


— 


— 


— 



Summary and Conclusions 

Preschool and kindergarten programs do affect young children’s development of conventional 
literacy skills as well as important emergent literacy skills. Results of the meta-analyses examining 
the overall effects of preschool and kindergarten programs across outcome measures revealed 
two main findings. The largest impact of the preschool and kindergarten programs was on the 
composite measure of readiness, indicating that they were highly effective in preparing children 
for school entry. 

The other main effect was a small to moderate impact of programs on spelling outcomes. 
Although the ES for spelling was smaller than that for readiness, it is significant that only 
kindergarten programs improved spelling. This might have resulted from the possibility that 
kindergarten programs were more likely to focus on spelling; such skills are rarely expected of 
preschoolers. Early spelling work is often proposed as a valuable component of beginning reading 
instruction because it involves the integration of phonemic awareness skills with AK. The studies 
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that contributed to this finding also included literacy-focused curricula, including teacher PD, 
further reinforcing the importance of these variables for effective implementation. 

A number of the other outcome variables had sufficient numbers of studies to allow for a meta- 
analysis of the results. For example, oral language had 12 studies, reading had nine studies, and 
AK and cognitive ability had four studies each. Yet, none of these outcome variables reached 
statistical significance. As has been explained earlier, in a meta-analysis magnitude of difference is 
as important as statistical significance. In this case, the oral language outcomes seem particularly 
modest, meaning that the range of preschool and kindergarten programs examined here would 
not be expected to exert much impact on this outcome. But contrast this with the large ES 
for reading outcomes; although, again, this difference did not reach statistical significance, the 
size of the difference is so large as to be of educational importance. These findings suggest that 
kindergarten and preschool programs can have an impact on childrens reading development. 

Additional analyses focused on identifying how the research design of studies of intervention 
effectiveness, such as preschool and kindergarten programs to affect early learning and school 
readiness, influence findings. In intervention-efficacy studies, it is important that sufficient 
controls are in place so that treatment effects can be separated from effects from no treatment or 
alternative treatment conditions. Analyses to compare the effects of RCT and QED found no 
differences between these research designs in studies of oral language outcomes, but there were 
differences between designs in studies of reading outcomes. The RCT reflected greater impacts 
for reading outcomes, although these findings may also reflect differences in whether teacher PD 
was included in the study. These findings suggest a need in future research for the characteristics 
of preschool and kindergarten programs to be explicitly compared. 

The most commonly measured outcome in all of the NELP intervention categories was oral 
language. Nine of the 33 preschool and kindergarten program studies included a composite measure 
of oral language skills, a measure of vocabulary, or both. The estimated ESs for programs on oral 
language and for vocabulary tended to be small, and these effects were not statistically reliable. 

The impacts of three types of preschool or kindergarten program characteristics were examined: 
literacy-focused curricula, PD for teachers, and PI. The presence of literacy-focused curricula 
and the availability of PD for teachers both strongly affected the reading outcomes for children 
in kindergarten programs. However, with the studies’ inclusion of both literacy-focused curricula 
and PD for teachers, it is impossible to separate the effects of the curriculum from the provision 
of teacher PD. Additionally, the studies contributing to this finding all focused on kindergarten 
children only; there is a clear need for research that examines such efforts with preschool children. 

Studies involving preschool and kindergarten programs with PI did not yield significant findings 
or sizable effects. Such findings had not been expected because of the reported effectiveness of 
high-profile preschool and kindergarten programs with strong PI (e.g.. Abecedarian project, 
Chicago Child-Parent Center Study, Head Start, and the Perry Preschool Project). It appears that, 
although PI in preschool or kindergarten programs has been strongly encouraged in the field, the 
specific impacts of such PI on early literacy outcomes have not been widely studied, and there 
is not yet a clear, empirically proven best way to use this involvement toward improved literacy 
performance for young children. 
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There is great interest in the impact of instructional programs on the learning of different racial, 
ethnic, linguistic, and economic groups of children. The data on preschool and kindergarten 
programs simply were not adequate to permit this kind of analysis. Future research will need to 
explore this issue more directly. 

Future Research 

More research is needed in several areas. For instance, very few experimental or quasiexperimental 
studies included children who are English-language learners or who have special education needs. 
The incidence of such children is increasing in early childhood classrooms, so it is important that 
there be an evidence base from which guidance can be sought on program effectiveness. 

Few studies have included infant or toddler programs as a focus of research for the promotion of 
early literacy skills. Although the studies published from the Abecedarian project contributed to 
findings reported here, there were few studies of very young children. More research in this area 
could identify the features and strengths of infant and toddler programs for improving literacy 
outcomes. Such studies would need to be longitudinal in design so that impacts of infant and 
toddler programs and concurrent assessments of the infants’ skills can be related to later measures 
of emergent literacy and conventional literacy skills. 

Also, there were fewer studies of preschool programs than of kindergarten programs. This 
mismatch stands in stark contrast to the increasing emphasis on the expansion of preschool 
enrollment toward making children more academically ready for kindergarten and first grade. 

An evidence base is needed to define and measure the benefits of preschool program attendance 
and especially to gauge progress toward the National Goals for Education (1994, 1998) that, by 
the year 2000 “every child will start school ready to learn.” Such an evidence base is needed to 
better understand the types of early literacy skills that are affected by preschool and kindergarten 
programs. There is a need for more studies of preschool programs. 
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Appendix 6.A. Primary Coding of Preschool and Kindergarten Program 
Interventions with Oral Language and Reading Outcome Measures 



Study 


Type of 
Study^ 


Age*' 


SES" Ethnic'* 


Population 

Density" 


Banerji, M. (1988). Longitudinal effects of retention and 
promotion in kindergarten on academic achievement. Florida 
Journal of Educational Research, 20(1 ), 59-72. 


2 


2 


2 1 


3 


Bergan, J. R., Sladeczek, 1. E., & Schwarz, R. D. (1 991 ). Effects 
of a measurement and planning system on kindergartners' 
cognitive development and educational programming. American 
Educational Research Journal, 20(3), 683-714. 


2 


2 


1 4 


2 


Brown, V. (1 992). Drama and sign language: A multisensory 
approach to the language acquisition of disadvantaged 
preschool children. Youth Theatre Journal, 6(3), 3-7. 


2 


1 


1 4 


3 


Carlson, C. D., & Erancis, D. J. (2002). Increasing the reading 
achievement of at-risk children through direct instruction: 
Evaluation of the Rodeo Institute for Teacher Excellence (RITE). 
Journal of Education for Students Placed at Risk, 7(2), 141-166. 


1 


2 


— 4 


3 


Erazier, J. A., & Morrison, E (1998). The influence of extended- 
year schooling on growth of achievement and perceived 
competence in early elementary school. Child Development, 
66(2), 495-517. 


2 


2 


— 4 


1 


Elurry, J., Sylva, K., & Riley, J. (1 999). Evaluation of a focused 
literacy teaching programme in reception and year 1 classes: 
Child outcomes. Rritish Education Research Journal, 25(5), 
637-649. 


2 


1 


— 4 


3 


Innocent!, M. S., Elollinger, P. D., Escobar, C. M., & White, K. 

R. (1993). The cost-effectiveness of adding one type of parent 
involvement to an early intervention program. Early Education 
and Development, 4(4), 306-326. 


1 


1 


1 4 


1 


Jenkins, J. R., Odom, S. L, & Speltz, M. L. (1 989). Effects 
of social integration on preschool children with handicaps. 
Exceptional Children, 55(5), 420-428. 


2 


1 


— 4 


3 


Jenkins, J. R., Speltz, M. L., & Odom, S. L. (1985). Integrating 
normal and handicapped preschoolers: Effects on child 
development and social interaction. Exceptional Children, 52(1), 
7-17. 


2 


1 


— 1 


3 


McGill-Eranzen, A., Yokoi, L., Brooks, G., & Allington, R. L. 
(1999). Putting books in the classroom seems necessary but not 
sufficient. Journal of Educational Research, 55(2), 67-74. 


1 


2 


3 4 


1 
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Mean ES for Primary 
Outcome Variables 
Oral Language Reading 
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0.27 
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2 


2 




0.04 


2 
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2 


2 


0.42 


— 




1 


1 


2 




2.90 


1 
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2 




0.37 


1 
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1 
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- 0.25 


0.11 




2 


2 


1 


- 0.03 




2 


2 


2 


2 


0.09 


— 




2 


2 


2 


- 0.58 




1 


1 


1 


1 


0.22 


1.18 
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Study 


Type of 
Study^ 


Age*' 


SES" Ethnic* 


Population 

Density" 


Ramey, C. I, & Smith, B. J. (1976). Assessing the intellectual 
consequences of early intervention with high-risk infants. 
American Journal of Mental Deficiency, 81{A), 31 8-324.' 


1 


1 


— 4 


3 


Roberts, J. E., Rabinowitch, S., Bryant, D. M., Burchinal, M. R., 
Koch, M. A., & Ramey, C. T. (1 989). Language skills of children 
with different preschool experiences. Journal of Speech & 
Hearing Research, 32, 773-786. 


1 


1 


1 2 


3 


Rodriguez, J. L., Diaz, R. M., & Espinoza, L. (1995). The impact 
of bilingual preschool education on the language development 
of Spanish-speaking children. Early Childhood Research 
Quarterly, 10, 475-490. 


2 


1 


1 3 


3 


Shepard, L. A. & Smith, M. L. (1987). Effects of kindergarten 
retention at the end of first grade. Psychology in the Schools, 
24, 346-357. 


2 


2 


— 4 


3 


Sparrow, S. S., Blachman, B. A., & Chauncey, S. (1 983). 
Diagnostic and prescriptive intervention in primary school 
education. American Journal of Orthopsychiatry, 55(4), 
721-729. 


2 


2 


1 4 


3 


Winsler, A., Diaz, R. M., Espinoza, L., & Rodriguez, J. L. (1 999). 
When learning a second language does not mean losing 
the first: Bilingual language development in low-income, 
Spanish-speaking children attending bilingual preschool. Child 
Development, 70(2), 349-362. 


2 


1 


1 4 


1 



‘RCT= 1,QED = 2. 

* Preschool or younger = 1, kindergarten = 2. 

' Low SES = 1, not low SES = 2, mixed SES = 3. 

'' Ethnicity: Caucasian = 1, African American = 2, Elispanic = 3, mixed = 4. 

‘ Population density: Urban = 1, mixed = 2, unknown = 3. 

' Vocabulary = 1, language composite = 2. 

® Literacy focus = 1, not literacy focus = 2. 

'' PD = I, not PD = 2. 

' PI = I, not PI = 2. 

' Other reports of this study contributed values to the overall ESs (Campbell, Ramey, Pungello, Sparling & Miller-Johnson, 2002; Campbell, Pungello, 
Miller-Johnson, Burch Inal, & Ramey, 2001; Ramey, Campbell, & Ramey, 1993; Campbell & Ramey, 1335; Campbell & Ramey, 1394; Ramey & 
Campbell, 1984; Ramey & Gowen, 1984; Ramey, Yeates, & Short, 19841. 
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Outcome Variables 
Oral Language Reading 
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- 0.03 


0.56 
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2 


1 


0.07 




2 


2 
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2 


0.38 
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2 


2 


2 


— 


0.46 


1,2 


1 


2 
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0.86 


0.85 


1,2 


2 


2 
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0.43 
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Introduction 

The National Early Literacy Panel (NELP) found that enhanced language learning should be a 
valued outcome when evaluating literacy-improvement efforts for young children. Of course, 
improvements in language learning may result incidentally from any early learning interventions 
(and those improvements are considered in the other chapters), but this chapter looks specifically 
at the effectiveness of interventions designed to explicitly and directly improve young childrens 
language skills, in terms of vocabulary development, syntactic sophistication, listening 
comprehension, and other similar aspects of language development. 

Three main questions are explored in this part of NELP s report: 

1. Did language-enhancement programs improve early literacy learning outcomes, including 
early learning outcomes important to later school-age reading? 

2. Did childrens demographic characteristics (e.g., socioeconomic status [SES], age, race) 
moderate the effects of a language-enhancement instruction? 

3. Were some kinds of language-enhancement instruction more effective than others? That is, 
were particular programs or implementation characteristics related to effectiveness? 



Description of the Language-Enhancement Studies 

The studies of language-enhancement interventions used various outcome measures to evaluate 
the effectiveness of these approaches. All of these studies included some measure of oral 
language development — most often a vocabulary measure — while others evaluated the effects 
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of language-enhancement efforts on phonemic awareness; cognitive ability; decoding; memory; 
print knowledge; rapid automatic naming (RAN); general readiness; and reading. No studies 
evaluated alphabet knowledge (AK), spelling, visual motor skills, or writing. Although these 
studies considered many different learning outcomes, there were usually too few studies to allow 
for analysis of the overall impact of language interventions on these variables (there had to be 
three studies that measured a particular construct to allow the results to be meta-analyzed). Table 
7. 1 includes the average effect sizes (ESs) presented in alphabetical order, numbers of studies, and 
significance of the interventions on the various outcomes. 



Table 7.1. Estimates of Effect Sizes Across Outcome Domains for Language-Enhancement Interventions 



Dependent Variable 


Fixed ES 


Random ES 




95% Cl 


Noi Studies 


pfor ES 








Lower 

Bound 


Upper 

Bound 






Cognitive ability 


0.85 


0.85 


0.27 


1.42 


1 


0.004 


Oral language 


0.61 


0.63 


0.42 


0.84 


19 


0.0001 


PA 


0.55 


0.57 


0.01 


1.14 


2 


0.05 


Print knowledge 


0.81 


0.81 


0.20 


1.41 


1 


0.009 


RAN 


0.54 


0.54 


-0.05 


1.13 


1 


0.075 


Readiness 


0.62 


0.62 


0.08 


1.16 


1 


0.024 


Reading 


0.20 


0.36 


-0.38 


1.10 


2 


0.343 



Note: Cl = Cl based on random-effect model. 



To be included in the analyses reported in this chapter, studies had to consider the effectiveness 
of some instructional effort implemented to improve young childrens language ability and skills. 
The studies included here had to meet a priori criteria that the panel established. The studies 
had to be carried out with children from birth through age 5 (or kindergarten) and had to use 
a group design, meaning that it compares the outcomes of an intervention with the results 
obtained from a comparison or control group. The comparison condition would include a group 
of children who were receiving no service or teaching beyond that routinely or traditionally used 
by the school or early childhood program, or an alternate treatment, while the experimental 
condition would have received the language-enhancement intervention. 

This design could be an experiment (meaning that the children were randomly assigned to the 
instruction) or a quasiexperiment (meaning that intact groups were assigned to the treatments; 
when this approach was taken, the study had to provide evidence that these groups were actually 
equal at the beginning of the study). A comparison of the experimental (randomized control trial 
[RCT]) versus quasiexperimental design (QED) studies in this category resulted in no differences 
in treatment effectiveness as a function of study design (Q[l,17] = 0.09, p = 0.76) (see Table 7.2). 
The studies had to provide a description of what was actually implemented with the children. 
Experiments of short-term trials, such as a single lesson, were excluded. Einally, the study had to 
measure the effects of the language-enhancement program with valid, reliable instruments that 
assessed childrens early literacy or language skills. 
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Table 1.2. Effect Sizes for Oral Language Outcomes Based on Type of Study Oesign 



Study Design 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








RCT 


0.65 


0.15 


0.35 


0.94 


U1 


9 


0.0005 


QED 


0.59 


0.12 


0.35 


0.82 


4.89 


10 


0.0001 



Nineteen studies were identified for inclusion in the analysis of the effectiveness of language- 
enhancement instruction. Forty-seven percent of the studies were published in the last decade, 

32 percent in the 1980s, and 21 percent in the 1970s. Researchers, teachers, parents, or others 
carried out these interventions in a variety of contexts (e.g., home, classroom, center). The 
interventions were short or long term, of varying intensity, and with few or many children, and 
included typically developing children as well as children with an array of language difficulties. 

The 1 9 studies varied considerably in outcomes measured, intervention durations, and ages of 
the children. About 70 percent of the studies included preschoolers or kindergarten children, 
with the rest considering the language growth of infants and toddlers (only one study included 
infants below one year of age). About half of the studies involved a relatively short intervention 
(less than 10 weeks), and, of those with longer interventions, the length was still usually no 
more than a few months, with a couple lasting for an entire school year. About 40 percent of 
the studies focused on children with language and learning delays. The sample size of the groups 
under study was often small, with fewer than 20 children (58 percent), and never exceeded 30 
children in each group. Most of the studies used random assignment of children to conditions 
(68 percent), with outcomes measured soon after the end of the intervention (79 percent). Only 
four of the 1 9 studies evaluated sustained effects at some later point after the completion of the 
intervention. The person administering the intervention ranged from a researcher or clinician 
(53 percent of the studies) to teachers (26 percent) or parents (16 percent), and, in one study, a 
computer administered the intervention. 

To measure the interventions’ effectiveness on children’s learning, a broad range of outcomes was 
included in these 19 studies. These are summarized in Table 7.3. 



Table 7.3. Oistribution of Outcome Measures Used in 19 Language-Enbancement Studies 



Outcome Assessments 


Percentage 
of Studies 


Language output (e.g., mean length of utterance, frequency of word use) 


15 


Gains in specific words or grammatical structures 


10 


Composite language scores 


26 


Complexity of multiword utterances 


26 


Listening comprehension 


16 


Literacy outcomes 


15 



There was a great deal of variability across the 1 9 studies in the type of intervention 
implemented. In general, interventions differed on such factors as amount of direction or 
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structure provided, the social context of the intervention, feedback to the child, and the type 
of language skill targeted for change. A typical intervention evaluated here might be referred 
to as focused-stimulation interventions (26 percent) . These were usually conducted within a 
naturalistic context in which the child heard specified language input (e.g., vocabulary, question 
types) often in game-like or play activities within their daily routines. Another frequent approach 
had children engaged in language activities, such as responding to wh questions or talking 
about similarities and differences in pictures (21 percent). Two other categories of language 
interventions were similar in the direct training of components of language, such as phonology 
(16 percent) or sentence structure (16 percent). Some studies did not easily fit into any of these 
categories. For example, only single studies examined the following approaches: the use of 
computer feedback to train vocabulary; building language through motor exercises; and building 
listening comprehension through exposure to stories read aloud. 



Do Language-Enhancement Interventions Improve Children's Language and 
Literacy Learning? 

The studies that looked at oral language development outcomes were grouped into three 
overlapping clusters for analysis. The first cluster, general oral language enhancement, included 
any measures of oral language, and this cluster included all 1 9 studies. A second cluster of 
eight studies, language composite, was drawn from these 1 9 studies and looked at composite or 
general measures of oral language development. Finally, a third group of 10 studies, oral language 
(vocabulary enhancement), focused specifically on vocabulary improvement alone. 

General Oral Language Enhancement as a Function of Language Intervention 

These 1 9 studies attempted to improve young children’s performance on a wide variety of 
oral language outcomes, including expressive or receptive vocabulary skills and grammatical 
development. The interventions were delivered in differing ways but usually in a small-group 
format. Parents, teachers, graduate trainees, speech-language clinicians, or trained home visitors 
delivered the interventions. These interventions were varied and included efforts to teach 
specific words, phonology, or morphosyntax, incidental teaching, enriched play experiences, 
and encouragement of creative thinking. Children with and without language problems were 
included, as were gifted kindergarten children and children in low- and middle-income families. 

The evaluation of language-enhancement interventions across these 1 9 studies showed that 
such interventions successfully improved children’s oral language development. The average ES 
for these 19 studies is 0.63 (using a random-effect model) (Q[l,17] = 1.08, p = 0.30), which is 
considered to be a moderate-sized effect. 

Oral Language (Language Composite) Enhancement as a Function of Language Intervention 

Eight studies contributed to the analysis of a mixed set of language outcomes (hence the term 
language composite). Among these, children with language delays or atypical communication 
skills were included in four of the studies, and toddlers or preschoolers were included as 
subjects in six of the studies. The interventions varied considerably, from focused or direct 
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training methods to training contextualized in adult-child interactive play or storybook-reading 
sessions to a motor-skill or physical-education context to which enriched language was added. 

For example, an interactive, child-centered stimulation program delivered by speech-language 
pathologists and focusing on vocabulary expansion and two- and three-word combinations 
was the enhancement delivered in one study of late-talking 2 1 - to 30-month-olds. In a second 
study of children with language delays or deviant communication skills, adult-child dyads with 
carefully scripted adult roles moved from imitation of child play toward more mature cooperative 
interactions, thus promoting an interpersonal context for communication instead of one directed 
more pointedly at speech production and comprehension. The comparison group received a 
more traditional, language-focused intervention. A third study provided language-enhanced 
physical-education activities for the treatment group, while the comparison group engaged in 
physical-education activities without language enhancement, with children in special education, 
typical prekindergarten and Head Start prekindergarten classes, in 24 sessions in an eight-week 
time frame. Yet another study varied instructional-unit size for kindergartners in the training of 
listening comprehension, using story reading in each intervention session, and comparing 1:1, 

1:7, and 1:15 teacher-to-child ratios. Although diverse in their intervention methods, agents, 
target areas of language enhancement, and rationales, the studies share the characteristic of 
casting a rather broad net of assessments as outcomes of interest. Virtually all of the studies were 
conducted in a center-based or school-based context, with the exception of one reporting that 
the enhancement sessions took place uniformly in one locale for each child, either at the child’s 
preschool or at home. The evaluation of language enhancement versus control across these eight 
studies yielded a significant result for the dependent measure, oral language (language composite) 
(Q[l,6] = 14.25104, p = 0.046). It is therefore worthwhile to report the measures represented in 
the composite group. These included measures of expressive vocabulary, oral language, verbal IQ, 
listening comprehension, language skills (not otherwise specified), phonemic awareness, concept 
of word, memory, oral-expression composite, RAN graphological and RAN nongraphological, 
reading comprehension, and visual motor skill. 

Oral Language (vocabulary enhancement) as a Function of Language Intervention 

The 10 studies included in this cluster were an array of language enhancements, usually 
delivered in small-group format in several sessions over several weeks, and almost all guided 
by teachers, graduate trainees, or speech-language clinicians. Two of the studies used parents 
as interventionists, and one employed computer-based training of vocabulary. The focus of 
language enhancement ranged from specific target-word learning to incidental teaching to 
encouragement of enriched play experiences or enhancement of creative thinking to training 
via phonological intervention or morphosyntax intervention. The oral language and vocabulary 
outcomes included expressive or receptive vocabulary skills and additional oral language abilities. 
Children with and without language problems were sampled in the mix of 10 studies, as were 
gifted kindergartners. The evaluation of language enhancement versus control across the 1 0 
studies yielded a nonsignificant result for the dependent measure, oral language-vocabulary 
(Q[l,8] = 5.55, p = 0.70). Again, this finding is limited by the strict inclusion criteria applied to 
all studies examined in the NELP report and by the intervention versus no-treatment comparison 
methodology required for this analysis. See Table 7.4 for a comparison of outcomes by type of 
language measure used (simple vocabulary measures versus composite measures of language). 
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Table 7.4. Effect Sizes of Oral Language Interventions on Measures of Simple Vocabulary and Composite 
Measures of Oral Language 



Measures 


Mean ES 


SE 


95% Cl 


t 


Na 


P 








Lower 

Bound 


Upper 

Bound 








Vocabulary 


0.54 


0.13 


0.28 


0.79 


4.14 


10 


< 0.0002 


Composite 


0.80 


0.22 


0.37 


1.22 


3.68 


8 


< 0.0002 



One of the 19 studies included in this analysis contributed both measure and is reflected in the total n. 



Even though it is impossible to provide further analysis of those outcome measures that 
were used in fewer than three studies, it is important to note that various non-oral language 
outcomes were examined in several studies and often with good results. For example, two studies 
considered the impact of oral language interventions on childrens phonological awareness (PA) 
and found significant improvement. Similarly, there were significant and sizable gains evident in 
individual studies that considered cognitive ability, print knowledge, and reading readiness. With 
more language-intervention studies that include these types of outcome measures in the future, it 
will be possible to determine whether other aspects of literacy-related learning are enhanced. 



Which Children Benefited from These Language-Enhancement Interventions? 

Language-enhancement interventions improved childrens language development, but some children 
may have benefited more than others from such interventions. The demographic information drawn 
from these studies was used to determine whether the results differed across groups of children. The 
demographic features that could be considered were age (Q(l,17) = 1.08, p=.30), ethnicity (Q[l,17] 
= 0.47, p = 0.49), SES (Q[2,7] = 3.40, p = 0.18), and population density (Q[3,15] = 0.04, = 

0.99). Tables 7.5, 7.6, 7.7, and 7.8 summarize these results and show that ES estimates did not differ 
significantly between groups for any of these demographic categories. 



Table 7.5. Effect Sizes for Oral Language Outcome for Language-Enbancement Interventions Based on Age 
of Cb I Idren in Study Sample 



Age 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 


Upper 














Bound 


Bound 








Prekindergarten 


0.71 


0.13 


0.45 


0.96 


5.37 


13 


0.00001 


Kindergarten 


0.51 


0.14 


0.25 


0.78 


3.78 


6 


0.002 



Table 7.6. Effect Sizes for Oral Language Outcome for Language-Enbancement Intervention Based on 
Ethnicity of Children in Study 



Race or Ethnicity 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








African American 


0.42 


0.30 


-0.17 


1.00 


1.40 


2 


0.18 


Mixed or unknown 


0.63 


0.10 


0.44 


0.83 


6.37 


17 


< 0.00002 
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Table IJ. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on 
Socioeconomic Status of Study 



SES 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Low 


0.36 


0.25 


-0.13 


0.85 


1.42 


3 


0.20 


Not low 


0.93 


0.21 


0.52 


1.34 


4.42 


5 


0.003 


Mixed 


0.48 


0.36 


-0.11 


1.18 


1.34 


2 


0.22 



Table 7.8. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on 
Population Oensity of Study Sample 



Population Density 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Rural 


0.62 


0.26 


0.10 


1.13 


2.35 


2 


0.033 


Urban 


0.62 


0.17 


0.29 


0.95 


3.71 


6 


0.002 


Suburban 


0.54 


0.39 


-0.21 


1.30 


1.40 


1 


0.181 


Unknown 


0.61 


0.13 


0.35 


0.88 


4.56 


10 


0.0004 



It should be noted, however, that there were serious limitations to these analyses. For instance, 
two of the studies focused on African American children, while the other 17 studies considered 
mixed groups of children or failed to provide information about ethnicity. Similarly, nine of 
the studies did not report adequate data to categorize SES. It is possible that there would be 
differences in the effectiveness of these kinds of interventions for different types of children, but 
this has not been reported or explored in the original studies enough to allow for a convincing 
and reliable determination of this at this time. 



Were Particular Types of Language-Enhancement Interventions More 
Successful Than Others? 

By analyzing specific characteristics in the interventions examined in the original studies and 
linking these characteristics to the ESs for those studies, it is possible to determine whether there 
are particular types of interventions that will be more successful than others. The identification of 
these intervention differences is limited by the adequacy of the descriptions of the interventions 
in the 1 9 studies. In addition, the studies frequently overlapped when grouping for these 
characteristics, making it difficult to determine independent effects of the interventions with the 
various outcomes. 

Are Interventions That Target Children Younger Than Three Years Old More Effective 
Than Those with Older Children? 

Eour intervention studies tested the effectiveness of a language intervention for children younger 
than three years old; three included toddlers (25.6 to 31 months), and one targeted infants 
(9-15 months). These four interventions varied somewhat, but all were toy centered, three 
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were child directed with an emphasis on giving language stimulation in response to the child’s 
interest, and the one with infants involved provision of different approaches to encourage vocal 
sound and word approximations. The four interventions ranged in duration from one to three 
months and so were relatively brief in nature. These four studies were contrasted with the other 
15 interventions that had targeted children older than three years of age (range 3.5 to five years) 
These 15 interventions also varied greatly on many dimensions (e.g., duration, intervention 
approach, person providing the intervention). Significant differences were found between 
the two groups of studies (Q [1,17] = 3.74, p < 0.05) with greater effectiveness found for the 
interventions that included children younger than three years of age. These results suggest that 
intervening earlier versus later is advantageous for enhancing children’s language development. 
Table 7.9 summarizes the results of this comparison. 



Table 7.9. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on Age 
of Children in Study Sample 



Age 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Prekindergarten 


0.54 


0.10 


0.34 


0.74 


5.32 


15 


<0.0001 


Birth to three years 


1.07 


0.26 


0.57 


1.58 


4.18 


4 


0.0006 



Do Interventions That Are Play-Based Provide Superior Results over Those 
Contextualized in Other Types of Learning Settings? 

The comparison of studies in which the intervention was more fully seated in a play-based 
activity with studies in which the intervention was not particularly play based did not result 
in significantly different outcomes (Q[l,15] = 0.24, p = 0.63). Six of the studies focused on 
interventions that were based in play activities, while 1 1 of the studies used other types of learning 
settings. Studies included in the play-based interventions were those that had toys available for 
the child and opportunities for the child to engage with toys in ways that he or she chose. For 
example, one intervention situated the language-enrichment exposure in physical-activity sessions 
in preschool, one used an adult play tutor to facilitate language enrichment during play, and, 
for one study, a toy demonstrator modeled rich verbal-stimulation techniques for the parent in 
demonstrating interactive play with the child at home. The nonplay interventions may or may 
not have included toys but did not allow the child to engage with toys on his or her own. For 
example, in one of these nonplay interventions, vocabulary training was done through computer 
story exposure and follow-up interactive computer games, while another used training conducted 
by teachers in generating questions. Two of the 1 9 studies were excluded from this analysis because 
there was not enough information to determine whether the intervention fit better as a play-based 
intervention. Table 7.10 summarizes the results of this comparison. 
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Table 7. 10. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on 
Whether the Intervention Was Play Based 



Play Based 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








No 


0.54 


0.12 


0.32 


0.78 


4.71 


11 


0.000 


Yes 


0.65 


0.18 


0.30 


1.01 


3.57 


6 


0.002 



Does the Intervention's Effectiveness Vary as a Function of the Language Status of the 
Children in the Study? 

Among the 19 studies, five included samples of children with language impairment, and five 
included samples of children with typically developing language skills for age (see Table 7.1 1). 
The other nine studies included a mixed sample of children or the language status of the children 
was not indicated, so these studies were excluded from this analysis. The types of impairments 
differed in the affected samples; for example, one study included a sample with expressive- 
vocabulary delay, another focused on children with delay in grammatical development, and one 
study included both speech and language impairments. No significant difference was found in 
intervention effectiveness as a function of the language status of the samples studied (Q[l,8] = 
1 . 52 ,;) = 0 . 22 ). 



Table 7. 1 1. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on 
Language Status of the Children in the Study Sample 



Language Impairment 


Mean ES 


SE 


95% Cl 


t 


P 


n 








Lower 

Bound 


Upper 

Bound 








No 


0.52 


0.16 


0.21 


0.85 


3.21 


0.012 


5 


Yes 


0.83 


0.46 


0.46 


1.20 


4.40 


0.002 


5 



Does the Effectiveness of Language Interventions Depend on the Agent (e.g., teacher, 
parent, computer) Who Delivers It? 

There were inadequate numbers of studies to make comparisons with regard to intervention 
agents. It was not possible, for instance, to determine whether teachers were as effective as 
speech-language pathologists. Some studies involved both parent and professionals as agents of 
intervention. However, there were adequate numbers of studies to compare teachers to parents. 
Three of the studies used teachers as interventionists, while four used parents. All three of the 
teacher-interventionist studies took place in kindergartens, without particular note of language 
delay or impairment in the samples studied; two of these included explicit teacher training in 
the program package or method of question generation that was the target of intervention. In 
the third, preservice teachers conducted the intervention sessions by reading prepared stories and 
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instructions for the questions asked about the stories. In contrast to the studies using teachers as 
agents of intervention, those that employed parents as interventionists included children both at 
and younger than kindergarten age, with half of the four studies including samples of children 
with language difficulties or delays. The comparison between intervention agents — teacher 
versus parent — yielded no significant difference in outcomes (Q[l,5] = 0.42, p = 0.52). It did 
not seem to matter who delivered the interventions, as children benefited in either case. Again, 
the small study set in this contrast limits its utility, as does the marked differences in the types 
of interventions being implemented by teacher versus parent as agent. Table 7.12 includes a 
summary of the results of this comparison. 



Table 7. 12. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on 
Agent of Intervention 



Agent 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








Parent 


0.64 


0.21 


0.21 


1.05 


2.96 


4 


0.032 


Teacher 


0.83 


0.22 


0.41 


1.25 


3.85 


3 


0.012 



Are Interventions That Are Structured Such That Feedback Is Given to the Child After He 
or She Responds More Effective Than Those That Do Not Provide Feedback? 

This question was possible to address because four of the intervention studies were similar in 
terms of providing some form of feedback to a child based on the type of response the child gave. 
These four studies were contrasted with eight intervention studies that did not give any form of 
systematic feedback following a child’s response. The other seven studies were excluded from this 
analysis because the interventions were not of the type in which feedback would be provided or 
it was not possible to determine whether feedback was given as a part of the intervention. This 
comparison was of interest because feedback might improve the child’s learning of the language 
concept. No significant differences were found in intervention effectiveness as a function of the 
provision of feedback following a child response (Q[l,10] = 0.009, p = 0.92). Table 7.13 gives a 
summary of the results of this comparison. This result needs to be interpreted with a great deal of 
caution, as only four studies could be included in the provision-of-feedback category. Also, while 
these four studies were similar in providing feedback, the conditions under which feedback was 
provided and the type of feedback provided varied considerably. Thus, a more stringent test of 
this question would include more studies investigating the effectiveness of interventions that used 
very similar approaches in the provision of feedback following a child’s language response. 

In general, all four were designed such that the person giving the intervention was trained to 
provide further questioning (one versus multiple questions) or, in one study, a hint or cue to 
guide the child to make a correct response if he or she gave an incorrect response. One of the 
four studies had a highly structured teacher-directed approach and targeted a broad range of 
language skills across a full school year. In contrast, the other three targeted more specific areas of 
language development for one to four months. Two of these three promoted questioning skills. 
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one in a structured format with emphasis on repetition of modeled questions and one with a 
more naturalistic, object-centered approach. The third targeted receptive vocabulary development 
with a computer delivering the intervention. The eight contrast interventions used a child- 
centered approach in which the adult providing the intervention gave enriched language input in 
reciprocal interactions. In all but one study, language stimulation was provided in the context of 
interactions centered on toys or pictures. In one study, language stimulation was provided in the 
context of physical activity. 



Table 7. 13. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on 
Type of Feedback Provided to the Child 



Type of Feedback 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








No feedback 


0.63 


0.17 


0.30 


0.96 


3.74 


8 


0.004 


Feedback 


0.66 


0.22 


0.23 


1.09 


3.01 


4 


0.013 



Are Interventions That Require a Child to Respond More Effective Than Those That Do 
Not Have This Requirement? 

Seven intervention studies were designed to require a child receiving the intervention to provide 
a response. In all seven studies, the child was required to respond in a range of ways, such as 
(1) answer a question (e.g., “What is this called?” while the interventionist points to or shows a 
picture or object), (2) repeat a modeled utterance, (3) describe characteristics of objects or ask 
questions about them, or (4) provide the name of a toy after hearing its name. Thus, for all seven 
of these interventions, the interventionist provided a certain degree of structure that might be 
expected to facilitate greater language learning. The seven studies varied on the length of the 
intervention. Three lasted about one month (four weekly sessions). Of the remaining four, two 
lasted four months and two lasted the full school year. Five were carried out with five-year-olds, 
and two interventions targeted two- and three-year-olds. 

Six intervention studies that did not require a child to give a response were contrasted with 
the seven studies that did. All six of these were also included as part of the eight studies in the 
previous section that did not provide feedback to a child’s response. In addition, six other studies 
could not be included in this analysis because descriptions of the interventions lacked sufficient 
information to determine whether the child was required to give a response or the interventions 
were of a different focus. The six interventions not requiring a response were child centered 
and toy or activity based (e.g., games, physical movement), and the interventionist followed the 
child’s lead without systematically requesting a response. Four included preschool- age children, 
and two conducted the intervention with two-year-olds. The length of the interventions ranged 
from three to seven months. When these two groups of studies were examined for differences 
in effectiveness, no significant differences were found (Q[l,l 1] = 0.35, p = 0.56). Table 14 
summarizes the results of this comparison. 
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Table 7. 14. Effect Sizes for Oral Language Outcome for Language-Enhancement Interventions Based on 
Required Child Response 



Response 


Mean ES 


SE 


95% Cl 


t 


n 


P 








Lower 

Bound 


Upper 

Bound 








None 


0.68 


0.19 


0.32 


1.05 


3.66 


6 


0.004 


Given 


0.55 


0.14 


0.27 


0.82 


3.85 


7 


0.003 



Summary and Conclusions 

Interventions designed to improve young childrens oral language skills have been effective. These 
interventions enhance oral language when it is defined as a diverse set of outcomes, such as 
expressive and receptive language skills, phonemic awareness, and verbal intelligence. 

It might be expected that oral language-enhancement interventions would work better with 
children who struggle with language or have some form of language impairment, but these 
analyses suggest this not to be the case, though differences might emerge from a larger sample 
of studies. Apparently, language-enhancement interventions provide a useful support to a broad 
range of children, including those with normal language functioning. Similarly, there were no 
differences in the effectiveness of these interventions for children on the basis of their SES, 
ethnicity, or the population density of where they live. 

The one difference that did seem to matter in the effectiveness of language-enhancement 
interventions concerned the childrens ages. Older children, between three and five years of age, 
did not get as big a language boost from these interventions as did the younger children. It would 
appear that intervening earlier rather than later is advantageous, although the exact process of this 
impact is not addressed here. 

Similarly, there seemed to be no key features to these interventions that consistently gave an 
advantage. All of these programs seemed to work. In fact, of the 1 9 studies, 1 8 had individual 
outcome effects that were moderate to large. 

There is a set of questions of both pressing practical significance and enormous theoretical 
importance that could not be addressed in these analyses. These are challenging questions 
that, if answered, would inform the field about teaching materials or strategies that provide 
maximum benefit for childrens language growth in the birth-to-five-year-old age range. For 
example, beyond the age-related results discussed in the preceding section, it would be useful 
to delineate successful approaches to language enhancement by age, slicing the age variable 
into more narrowly defined periods of one- to two-year intervals or evaluating a broader range 
of interventions by age. School and day-care settings often separate age groups in the birth-to- 
five-year-old population, making difficult the generalization of language-intervention findings 
obtained with the oldest children to the youngest. The lack of adequate numbers of studies for 
meaningful comparisons blocked analysis of this variable. Several other questions deserving of 
careful attention could not be addressed because a critical number of studies that met the criteria 
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to be included in the research synthesis was lacking or because information that would guide 
specific contrasts was missing. Among those questions are the following: 

• Is there benefit to the adoption of specific approaches to teaching in language 
interventions (e.g., direct instruction versus naturalistic or milieu-based interventions)? 

• Can we comment on the effectiveness of specific curricula developed for the birth-to- 
five-year-old population (e.g., computer software-based curricula, commercially available 
curricula with instruction delivered through teachers and curricular materials, researcher- 
mounted curricula delivered through teachers, parents, or researchers)? 

• Is there information on best practices for delivering language interventions for specific 
populations of children (e.g., children with language impairments, children who are 
English-language (or whatever the language of school instruction is) learners, children in 
low-income families)? 

• Does success vary as a function of the agent of intervention (e.g., researchers, speech- 
language pathologists, other professionals)? 

• Does outcome differ with the intensity of the intervention (e.g., frequency of 
applications per week, group size, group versus individual training)? 

• How shall we conceptualize the interaction of intervention strategy, frequency of 
application, and age group? 

Considerations for Future Research 

One of the most important outcomes of this endeavor is the identification of research areas that 
need much more attention that are related to the efficacy of language interventions for young 
children. The following six areas of research are suggested as a starting point for generating a 
better understanding of what interventions work and for which children, as well as the aspects of 
early language and literacy development that they enhance: 

• examinations of language curricula and programs addressing the ages at which they are 
most effective. These studies need to be conceptually guided and include a range of 
outcomes that the intervention might affect. 

• more replication studies of the interventions that show positive effects. These studies 
need to expand the cohorts of children studied in the original interventions to provide 
information on the interventions generalizability to the broadest range of children. 
Attention also needs to be given to such issues as intensity and agent of the intervention 
and training methods for those administering the intervention. 

• attention to large cohort studies that examine programs that might show efficacy in 
enhancing specific aspects of language development. These include expressive and 
receptive language for vocabulary, syntax, semantics, and pragmatic skills. 

• attention to the need for a more unifying terminology of characteristics of children at 
risk for language problems and those identified as language impaired. This research needs 
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to demonstrate how well-specified, evidence-based programs can address the language 
needs of different groups of children at risk. 

• more longitudinal research that provides information on the sustainability of the 
effectiveness of intervention programs. 

The importance of addressing these questions is clear, and the information we lack precludes 
making careful and precise statements to guide practice. While an unsatisfying conclusion, this 
is nonetheless a highly pertinent one; gaps in systematically collected data (that is, the studies 
meeting criteria for the evaluation of language interventions) leave us with only a sketchy 
response to extremely important curricular and intervention questions. There is clearly a need to 
prioritize research that can both guide practice and provide evidence-based options for the field. 

In light of the high prevalence of language delays and impairments identified in young children, 
as well as the needs of children who are English-language learners, this report highlights the need 
for a great deal more research in these areas. Current findings are often based on small samples 
of children, and, too often, the studies are not rigorously described. More research is needed 
using rigorously implemented and described experimental interventions, with sample sizes and 
sample characteristics that would facilitate generalization. It is noteworthy that only one study 
provided a longitudinal follow-up component that could help ensure that the program’s effects 
could sustain. There is evidence from the language-intervention studies examined in this report 
that addressing children’s language needs at earlier rather than later ages provides better results. 
Thus, it is imperative that we identify the enhancement approaches that teachers, parents, and 
clinicians need in order to give children the best opportunity to enter schools with the language 
skills they need to succeed. 
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Appendix 7.A. Primary Coding of Studies with Oral Language Outcome Measures 



Study 


Type of 
Study^ 


Age‘’ 


SES-^ 


Ethnic'' 


Population 

Density" 


Asbury, C. (1970). Section C: Some effects of training on verbal 
mental functioning in Negro pre-school children: A research 
note. Journal of Negro Education, 39{]), 100-103. 


1 


1 


2 


1 


4 


Benner, G. J., Trout, A, Nordness, P. D., Nelson, J. R., Epstein, 
M. El., Knobel, M., Epstein, A., et al. (2002). The effects of the 
Language for Learning program on the receptive language skills 
of kindergarten children. Journal of Direct Instruction, 2{2], 
67-74. 


2 


2 




2 


1 


Beveridge, M., & Jerrams, A. (1981 ). Parental involvement 
in language development: An evaluation of a school-based 
parental assistance plan. British Journal of Educational 
Psychology, 5/(3), 259-269. 


2 


1 


1 


2 


2 


Cassidy, A. M., & Vukelich, C. (1 977). The effects of group 
size on kindergarten children's listening comprehension 
performance. Psychology in the Schools, /4(4), 449-455. 


1 


2 


— 


2 


3 


Christie, J. E (1983). The effects of play tutoring on young 
children's cognitive performance. Journal of Educational 
Research, 76(6), 326-330. 


1 


1 


1 


2 


1 


Connor-Kuntz, E J., & Dummer, G. M. (1 996). Teaching across 
the curriculum: Language-enriched physical education for 
preschool children. Adapted Physical Activity Quarterly, 13, 
302-315. 


1 


1 


3 


2 


4 


Eey, M. E., Cleave, P. L., Long, S. El., & Elughes, D. L. (1993). 
Two approaches to the facilitation of grammar in children with 
language impairment: An experimental evaluation. Journal of 
Speech & Blearing Research, 36, 1 41 -1 57. 


1 


1 




2 


4 
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delays. Journal of Speech and Blearing Research, 39, 1274- 
1283. 


1 


1 


2 


2 


2 


Glaubman, El., Glaubman, R., & Ofir, L. (1997). Effects of self- 
directed learning, story comprehension, and self-questioning in 
kindergarten. Journal of Educational Research, 60(6), 361-374. 


1 


2 


2 


2 


2 


Elamilton, M. (1977). Social learning and the transition from 
babbling to initial words. Journal of Genetic Psychology, 130, 
211-220. 


2 


1 


3 


2 


2 
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Language 

Status' 
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Feedback" 
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Mean 

ES 


— 


2 


2 


— 


— 


— 


— 


0.42 


1 


2 


2 


2 


1 


1 


1 


0.81 


1 


2 


2 




2 






0.46 


2 


2 


2 


2 


1 


— 
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0.54 


1 


2 


1 


2 
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2 


0.14 


2 


2 


1 






2 


2 


0.13 


2 


2 




1 


2 






0.88 


1 


1 


1 


1 


2 


2 


2 


0.86 


2 


2 


2 


— 


1 


1 


1 


1.31 


— 


1 


2 


— 


— 


— 


— 


2.05 
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Type of 
Study^ 


Age** 


SES° Ethnic'' 


Population 

Density" 


Korkman, M. (1993). Preventive treatment of dyslexia by 
a preschool training program for children with language 
impairments. Journal of Clinical Child Psychology, 22{2], 
ni-m. 


2 


2 


— 2 


4 


Levenstein, P. (1970). Cognitive growth in preschoolers 
through verbal interaction with mothers. American Journal of 
Orthopsychiatry, 40(3), 426-432. 


2 


1 


1 1 


2 


Meador, K. S. (1994). The effect of synectics training on gifted 
and nongifted kindergarten students. Journal for the Education 
of the Gifted, ?0(1 ), 55-73. 


2 


2 


— 2 


2 


Robertson, S. B., SiWeismer, S. E. (1999). Effects of treatment 
on linguistic and social skills in toddlers with delayed language 
development. Journal of Speech. Language, and Hearing 
Research, 42 1234-1248. 


2 


1 


2 2 


4 


Segers, E., & Verhoeven, L. (2003). Effects of vocabulary 
training by computer in kindergarten. Journal of Computer 
Assisted Learning, 19, 557-566. 


2 


2 


— 2 


4 


Smith, C., & Eluck, M. (2000). (Re-)constructing pre-linguistic 
interpersonal process to promote language development in 
young children with deviant or delayed communication skills. 
British Journal of Educational Psychology 70, 369-389. 


2 


1 


— 2 


4 


Tyler, A. A., Lewis, K. E., Haskill, A., & Tolbert, L. C. (2003). 
Outcomes of different speech and language goal attack 
strategies. Journal of Speech, Language, and Hearing 
Research, 46(5), 1077-1094. 


1 


1 


— 2 


4 


Valdez-Menchaca, M. C., & Whitehurst, G. J. (1988). The 
effects of incidental teaching on vocabulary acquisition by 
young children. Child Development, 5S(6), 1451-1459. 


1 


1 


2 2 


4 


Valian, V, & Casey, L. (2003). Young children's acquisition of 
iv/r-questions: The role of structured input. Journal of Child 
Language, 36(1), 117-143. 


2 


1 


— 2 


4 



‘RCT= 1,QED = 2. 

‘ Preschool or younger = 1, kindergarten = 2. 

' Low SES = 1, not low SES = 2, mixed SES = 3. 

African American = 1, mixed or unknown = 2. 

‘ Population density: Rural= 1, urban = 2, suburban = 3, unknown = 4. 
' Vocabulary = 1, language composite = 2. 

® Younger than age 3=1, age 3 and older = 2. 

‘ Play= 1, not play = 2. 

' Language impaired = 1, not language impaired = 2. 

‘ Teacher = 1, parent = 2. 

‘ Feedback =1, no feedback = 2. 

' Response = 1, no response = 2. 
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